CAD electrical scheme identification method and system

Abstract

The invention relates to the technical field of electrical identification, and discloses a CAD electrical scheme identification method and system, and the method comprises the steps: obtaining an overall image of a CAD electrical scheme, carrying out the target detection of the overall image of the CAD electrical scheme, and obtaining an image segment of each electrical scheme; identifying the image segments by using an image identification model to obtain image feature vectors, and performing similarity calculation to obtain image similarity; identifying the image segment by using a multi-modal large model to obtain an image-text description feature, and extracting the text description feature by using a text embedding vector model to obtain a text description feature vector; carrying out similarity calculation to obtain text description similarity; and according to the image similarity and the text description similarity, calculating a category fusion score, and according to the category fusion score, performing category classification on the image segments. According to the invention, a batch, rapid and accurate CAD electrical scheme identification function is realized.

Description

Technical Field

[0001] This invention relates to the field of electrical identification technology, and in particular to a method and system for identifying electrical schemes in CAD. Background Technology

[0002] Most existing methods for identifying CAD electrical schemes rely on manual visual inspection, while some are based on geometric feature extraction and rule template matching. Manual visual inspection involves engineers or designers opening CAD drawings, relying on experience to manually classify each electrical scheme, and compiling the results into an Excel spreadsheet or internal bill of materials. This entirely manual process is time-consuming, susceptible to experience level fluctuations, and cannot be batch-processed; the workload increases rapidly with the number of drawings. Methods based on geometric feature extraction and rule template matching often only provide single feature extraction and template matching functions, still requiring significant manual intervention (such as filtering scheme areas). This is inefficient, error-prone, and fails to achieve batch, fast, and accurate processing, nor can it form an end-to-end process from DXF file input to electrical scheme list output. Furthermore, existing methods based on geometric feature extraction and rule template matching are highly sensitive to the drawing style, line width, and symbol shape of CAD drawings. Even slight deviations in drawing format can lead to incorrect identification, and differences in drawing habits among different suppliers or design teams can result in numerous misjudgments, severely lacking universality. However, manually setting rule templates and retraining models is required for each new electrical solution, which is inefficient and costly. In practical applications, electrical solutions are updated frequently. If manual setting of rule templates and retraining of models are required for each new solution, it not only consumes a lot of computing resources and time, but also affects business response speed and user experience.

[0003] Therefore, how to provide a solution that enables batch, fast, and accurate identification of CAD electrical schemes is a problem that urgently needs to be solved. Summary of the Invention

[0004] The purpose of this invention is to provide a CAD electrical scheme identification method and system to solve the above-mentioned technical problems in the prior art.

[0005] According to a first aspect of the present invention, a method for identifying CAD electrical schemes is provided.

[0006] The CAD electrical scheme identification method includes: Acquire the overall image of the CAD electrical scheme, and perform target detection on the overall image of the CAD electrical scheme to obtain image segments of each electrical scheme; The image fragments are identified using a pre-configured image recognition model to obtain image feature vectors. Based on the obtained image feature vectors and the image feature vectors of known electrical scheme categories, similarity calculation is performed to obtain the image similarity of the image fragments to each electrical scheme category. The image fragment is identified using a pre-configured multimodal large model to obtain text description features of the image fragment, and the text embedding vector model is used to extract the text description features to obtain text description feature vectors; the text description feature vectors are then compared with the text description feature vectors of known electrical scheme categories to obtain the text description similarity of the image fragment to each electrical scheme category. Based on the image similarity and the text description similarity, a category fusion score is calculated, and the image fragment is classified according to the category fusion score.

[0007] The process of performing target detection on the overall image of the CAD electrical scheme to obtain image segments of each electrical scheme includes: using a pre-trained YOLO target detection model to perform target detection on the overall image of the CAD electrical scheme to obtain the bounding box coordinates of each electrical scheme; and cropping the overall image of the CAD electrical scheme based on the obtained bounding box coordinates to obtain image segments of each electrical scheme.

[0008] The YOLO object detection model is optimized using a multi-task loss function, the formula of which is:

[0009] In the formula, Represents the multi-task loss function; This represents the bounding box regression loss, used to optimize the overlap between the predicted and the true bounding boxes. This represents the target confidence loss, used to determine whether there is an electrical scheme target in the grid or candidate box; To represent classification loss, used to correctly predict the category of the target; , , These represent the coefficients that balance the contribution of each part of the loss.

[0010] The image recognition model includes either a ResNet network model or a DINO network model, wherein both the ResNet network model and the DINO network model are optimized and trained using the ArcFace loss function.

[0011] The multimodal large model includes the Qwen-VL-7B multimodal large model.

[0012] The text embedding vector model includes a text embedding model trained using Qwen3-8B-Embedding.

[0013] The calculation methods for both the image similarity and the text description similarity are cosine similarity calculation methods.

[0014] The formula for calculating the category fusion score is as follows:

[0015] In the formula, Indicates the category fusion score; Maximum image similarity for each electrical scheme category; Textual description similarity for each electrical solution category; , representing the fusion weights determined on the validation set through grid search or Bayesian optimization.

[0016] The process of classifying the image fragment according to the category fusion score includes: comparing the calculated category fusion score with a predetermined fusion threshold; and if the comparison result shows that the category fusion score is greater than the fusion threshold, determining that the image fragment belongs to the electrical scheme category corresponding to the image similarity and the text description similarity.

[0017] According to a second aspect of the present invention, a CAD electrical scheme identification system is provided.

[0018] The CAD electrical scheme identification system includes: The target detection module is used to acquire the overall image of the CAD electrical scheme and perform target detection on the overall image of the CAD electrical scheme to obtain image segments of each electrical scheme; The image recognition module is used to recognize the image segment using a pre-configured image recognition model, obtain image feature vectors, and perform similarity calculation based on the obtained image feature vectors and the image feature vectors of known electrical scheme categories to obtain the image similarity of the image segment to each electrical scheme category; The text recognition module is used to recognize the image fragment using a pre-configured multimodal large model, obtain the text description features of the image fragment, and extract the text description features using a text embedding vector model to obtain a text description feature vector; the text description feature vector is then compared with the text description feature vectors of known electrical scheme categories to obtain the text description similarity of the image fragment to each electrical scheme category. The category classification module is used to calculate a category fusion score based on the image similarity and the text description similarity, and to classify the image fragments according to the category fusion score.

[0019] Specifically, when the target detection module performs target detection on the overall image of the CAD electrical scheme to obtain image segments of each electrical scheme, it uses a pre-trained YOLO target detection model to perform target detection on the overall image of the CAD electrical scheme to obtain the bounding box coordinates of each electrical scheme; based on the obtained bounding box coordinates, the overall image of the CAD electrical scheme is cropped to obtain image segments of each electrical scheme.

[0020] The YOLO object detection model is optimized using a multi-task loss function, the formula of which is:

[0021] In the formula, Represents the multi-task loss function; This represents the bounding box regression loss, used to optimize the overlap between the predicted and the true bounding boxes. This represents the target confidence loss, used to determine whether there is an electrical scheme target in the grid or candidate box; To represent classification loss, used to correctly predict the category of the target; , , These represent the coefficients that balance the contribution of each part of the loss.

[0022] The image recognition model includes either a ResNet network model or a DINO network model, wherein both the ResNet network model and the DINO network model are optimized and trained using the ArcFace loss function.

[0023] The multimodal large model includes the Qwen-VL-7B multimodal large model.

[0024] The text embedding vector model includes a text embedding model trained using Qwen3-8B-Embedding.

[0025] The calculation methods for both the image similarity and the text description similarity are cosine similarity calculation methods.

[0026] The formula for calculating the category fusion score is as follows:

[0027] In the formula, Indicates the category fusion score; Maximum image similarity for each electrical scheme category; Textual description similarity for each electrical solution category; , representing the fusion weights determined on the validation set through grid search or Bayesian optimization.

[0028] When classifying the image segment according to the category fusion score, the category classification module compares the calculated category fusion score with a predetermined fusion threshold. If the comparison result is that the category fusion score is greater than the fusion threshold, the module determines that the image segment belongs to the electrical scheme category corresponding to the image similarity and the text description similarity.

[0029] The technical solution provided by this invention may include the following beneficial effects: This invention, based on a target detection model, performs unified image processing and then inputs it into a trained image recognition model, a multimodal large-scale model, and a text embedding vector model. It then uses the image similarity output by the image recognition model and the text description similarity output by the multimodal large-scale model and the text embedding vector model to perform electrical scheme matching. This process requires no manual intervention, automatically completing the detection and recognition of electrical schemes. It achieves an end-to-end closed loop from CAD drawing import to electrical scheme recognition output, significantly improving recognition efficiency and reducing the error probability, thus realizing batch, fast, and accurate CAD electrical scheme recognition functionality.

[0030] Furthermore, by optimizing the multi-task loss function and the ArcFace loss function, each model can automatically learn the common features of electrical schemes, significantly reducing the dependence on the original drawing style. Even if there are differences in drawing style, line width, symbol shape, and other drawing methods, the electrical scheme can be accurately identified, greatly improving its versatility and maintaining a high recognition rate for CAD drawings drawn by different manufacturers, at different times, and by different personnel.

[0031] Meanwhile, when adding new electrical schemes, only image similarity and text description similarity are needed for scheme identification, eliminating the need for manual rule template setting / retraining. This achieves efficient system expansion and significantly reduces model maintenance costs and resource consumption. In practical applications, open set detection can be implemented based on image similarity and text description similarity, classifying samples dissimilar to known category features as "unknown," effectively distinguishing unknown electrical schemes and avoiding misclassification.

[0032] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0033] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0034] Figure 1This is a flowchart illustrating a CAD electrical scheme identification method according to an exemplary embodiment; Figure 2 This is a structural block diagram illustrating a CAD electrical scheme identification system according to an exemplary embodiment; Figure 3 This is a schematic diagram of the structure of a computer device according to an exemplary embodiment. Detailed Implementation

[0035] The following description and accompanying drawings fully illustrate specific embodiments described herein to enable those skilled in the art to practice them. Some embodiments may include or substitute parts and features of other embodiments. The scope of the embodiments herein encompasses the entire scope of the claims and all available equivalents thereof. Throughout this document, the terms “first,” “second,” etc., are used only to distinguish one element from another without requiring or implying any actual relationship or order between the elements. Indeed, a first element can also be referred to as a second element, and vice versa. Furthermore, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a structure, apparatus, or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a structure, apparatus, or device. Without further limitation, an element defined by the phrase “comprising one…” does not exclude the presence of other identical elements in the structure, apparatus, or device that includes said element. The various embodiments described herein are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments; similar or identical parts between embodiments can be referred to interchangeably.

[0036] The terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer" used in this document to indicate orientations or positional relationships are based on the orientations or positional relationships shown in the accompanying drawings. They are used solely for the convenience of describing the document and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In the description herein, unless otherwise specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two elements; they can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0037] In this document, unless otherwise stated, the term "multiple" means two or more.

[0038] In this article, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0039] In this article, the term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0040] It should be understood that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order constraint on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the diagram may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0041] The modules in the apparatus or system of this application can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0042] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0043] Figure 1 An embodiment of a CAD electrical scheme identification method of the present invention is shown.

[0044] In this optional embodiment, the CAD electrical scheme identification method includes: Step S101: Obtain the overall image of the CAD electrical scheme, and perform target detection on the overall image of the CAD electrical scheme to obtain image segments of each electrical scheme; Step S102: The image segment is identified using a pre-configured image recognition model to obtain an image feature vector. Based on the obtained image feature vector and the image feature vector of the known electrical scheme category, a similarity calculation is performed to obtain the image similarity of the image segment to each electrical scheme category. Step S103: The image segment is identified using a pre-configured multimodal large model to obtain the text description features of the image segment, and the text embedding vector model is used to extract the text description features to obtain the text description feature vector; the text description feature vector is compared with the text description feature vector of known electrical scheme categories to obtain the text description similarity of the image segment to each electrical scheme category. Step S104: Calculate the category fusion score based on the image similarity and the text description similarity, and classify the image fragment according to the category fusion score.

[0045] Figure 2 An embodiment of a CAD electrical scheme identification system according to the present invention is shown.

[0046] In this optional embodiment, the CAD electrical scheme identification system includes: The target detection module 201 is used to acquire the overall image of the CAD electrical scheme and perform target detection on the overall image of the CAD electrical scheme to obtain image segments of each electrical scheme; The image recognition module 202 is used to recognize the image segment using a pre-configured image recognition model, obtain image feature vectors, and perform similarity calculation based on the obtained image feature vectors and the image feature vectors of known electrical scheme categories to obtain the image similarity of the image segment to each electrical scheme category; The text recognition module 203 is used to recognize the image segment using a pre-configured multimodal large model, obtain the text description features of the image segment, and extract the text description features using a text embedding vector model to obtain a text description feature vector; and calculate the similarity between the text description feature vector and the text description feature vector of a known electrical scheme category to obtain the text description similarity of the image segment to each electrical scheme category. The category classification module 204 is used to calculate a category fusion score based on the image similarity and the text description similarity, and to classify the image fragments according to the category fusion score.

[0047] To facilitate understanding of the above technical solutions of the present invention, the following will provide a detailed description of the above technical solutions of the present invention from the perspectives of the specific training stage and the reasoning stage.

[0048] Training phase: First, data labeling and training set preparation are required: During the model training phase, the locations of electrical schemes in the images need to be labeled to construct a training dataset for object detection. Specifically, multiple electrical schemes in each circuit image are marked with rectangular bounding boxes, and all different types of electrical schemes are grouped into the category of "electrical scheme" for detection. The large number of labeled images obtained in this way can be used to supervise the model's learning of how to locate electrical scheme targets.

[0049] In this invention, the YOLO (You Only Look Once) object detection model is selected as the electrical scheme detection model. YOLO is a single-stage object detection algorithm that treats object detection as a holistic regression problem, capable of simultaneously predicting the bounding box position and class label of an object in an image in a single forward propagation. This type of model is characterized by its high speed and efficiency. Although the implementation details of different versions of the YOLO series vary, the core idea remains the same: using an improved convolutional neural network backbone to extract image features, coupled with an efficient detection head, to complete the prediction of object position and class in a single computation.

[0050] To obtain better initial features, the weights of a YOLO model pre-trained on a large dataset can be initialized. Subsequently, the pre-trained model is fine-tuned on a self-collected circuit image dataset to better adapt the model to specific circuit diagram scenarios.

[0051] During training, the YOLO model's network structure typically includes a convolutional neural network as the backbone, a feature pyramid or similar structure as the neck for fusing features at different scales, and several output layers as the head. This architecture allows the model to output detection results on feature maps at different scales, enabling simultaneous detection of targets of varying sizes. Each output prediction of the model contains three main pieces of information: (1) the coordinates of the bounding box (used to locate the electrical scheme), (2) the confidence level of the target's existence (indicating the probability that an electrical scheme exists within the predicted bounding box), and (3) the probability distribution of each category (in this scenario, since there is only one category, "electrical scheme," this is essentially equivalent to determining the probability of whether the region is an electrical scheme).

[0052] Since there is only one detection category, "Electrical Scheme", the detection head only needs to distinguish between the foreground (Electrical Scheme) and the background when classifying, which simplifies the problem to a binary classification (is it an electrical scheme or not an electrical scheme) judgment.

[0053] YOLO model training employs a multi-task loss function to simultaneously optimize localization and classification performance. Loss function This can be expressed as the sum of the losses of each part:

[0054] in, It is the bounding box regression loss, used to optimize the positional overlap between the predicted bounding box and the true bounding box; It is the target confidence loss, used to determine whether there is an electrical scheme target in the grid or candidate box (calculated using binary cross-entropy loss, which distinguishes between positive and negative samples). It is a classification loss, used to correctly predict the category of the target (in this scheme there is only one category, "electrical scheme"), therefore Primarily used to distinguish between foreground electrical schemes and background. , , These are the coefficients that balance the contribution of each part of the loss.

[0055] During actual training, the labeled circuit image is input into the YOLO model, which outputs predicted bounding box coordinates, target confidence, and class probability. Then, the difference between the model's prediction and the manually labeled data is calculated based on the aforementioned loss function, and stochastic gradient descent (SGD) or other optimization algorithms are used to iteratively adjust the model's parameters. As training progresses, the model continuously minimizes the loss function, with each part of the loss gradually decreasing. After multiple iterations until the loss converges, the YOLO model gradually learns to accurately locate all electrical components in complex circuit images.

[0056] Training and fine-tuning the YOLO model using the methods described above can significantly improve its ability to detect electrical scheme targets. The YOLO model was fine-tuned and trained using several circuit diagrams containing varying numbers of electrical schemes. Results show that the model achieved high detection accuracy on the validation set, reliably detecting various electrical schemes in the circuit diagrams and achieving precise localization and identification of electrical scheme targets. This demonstrates the effectiveness and robustness of using the YOLO model for electrical scheme detection.

[0057] In the image recognition part, the ResNet series of convolutional neural networks can be used as the feature extraction backbone of the electrical solution. The ResNet network is composed of multiple residual blocks stacked together. Each residual block contains convolutional layers and skip connections of identity mapping, thereby learning residual representations to alleviate the gradient vanishing problem of deep networks.

[0058] The model input first passes through a large convolutional kernel (e.g., 7×7, stride 2) and a downsampling pooling layer to extract low-level features and reduce the feature map size. The network then proceeds in four stages, stacking residual blocks sequentially. The number of output channels doubles at each stage (e.g., 64→128→256→512), and the feature map size is halved during stage transitions using convolutional or pooling layers with a stride of 2. This combination of numerous residual modules and skip connections allows ResNet to converge stably and extract high-level semantic features even with deep layers. At the network's end, global average pooling compresses the final feature map into a global feature vector in spatial dimension. This vector is then passed through a fully connected layer as the initial classification head, outputting feature representations or class logits (the raw numerical vector output from the last layer of the neural network, representing the model's unnormalized confidence score for each class).

[0059] In the electrical scheme recognition task, the original classification layer of the pre-trained ResNet backbone network is replaced with a new fully connected layer to output image feature vectors. Then, the entire network is fine-tuned and trained on a self-built electrical scheme image dataset.

[0060] In addition to using ResNet as the backbone of the image recognition model, the electrical scheme recognition module of this invention can also employ a more advanced visual Transformer architecture, such as the DINO model proposed by Meta AI. DINO is a self-supervised pre-trained Vision Transformer (ViT) model that has learned rich image feature representations on unlabeled image data. DINO uses ViT as its backbone: first, the input image is divided into fixed-size image patches, and each patch is embedded into a vector sequence through linear mapping, which is then input into a multi-layer Transformer encoder. Each Transformer layer consists of a multi-head self-attention mechanism and a feedforward network, extracting high-level semantic features layer by layer. After self-supervised training on a large-scale unlabeled image database, DINO's backbone network can generate high-quality feature representations with global semantics.

[0061] In this invention, a pre-trained DINO model can be used as the feature extraction backbone for electrical scheme recognition, with a new fully connected layer added on top to output image feature vectors. Fine-tuning is then performed using a method similar to ResNet: the electrical scheme image and its label are input into the model, and the DINO Transformer backbone and the new fully connected layer are jointly optimized. With limited training data, DINO's global modeling capabilities often achieve higher recognition performance than traditional convolutional networks.

[0062] Furthermore, this invention introduces the ArcFace loss function (additive angular margin loss) into the training of the image recognition model to improve model performance in the following aspects: Improving the ability to distinguish between different categories: By increasing the inter-class feature distance and decreasing the intra-class distance, feature confusion between different electrical scheme categories is reduced. Enhancing the model's open-set recognition ability: The features extracted by the model can remain clearly distinguishable from known categories even when the category is unseen, facilitating the detection of "unknown" categories during inference through threshold judgment. Obtaining a universal feature embedding representation: The model becomes a universal image encoder, outputting embedding vectors in a unit feature space, providing convenience for rapidly expanding to new categories.

[0063] The ArcFace loss function was initially applied to face recognition tasks. Its core idea is to add a fixed angular interval to the Softmax function classification. The algorithm maximizes inter-class margin and minimizes intra-class variance by using an angle as a metric, thereby improving discriminative power. Specifically, after normalizing both the feature vector and classification weight vector to a unit sphere, the angle between them is calculated. For each sample's true class, an angle is artificially added to the score of that class when calculating the Softmax function (i.e., an angular interval is added). This results in a smaller effective score, forcing the network to produce more extreme judgments to classify the sample into the correct class. This is equivalent to "pushing apart" features of different categories in a high-dimensional feature space.

[0064] During training, the image features extracted by ResNet / DINO are optimized using the ArcFace loss function: the term corresponding to the true class is adjusted by adding an angular interval. This approach allows the model to proactively increase the angular distance between feature vectors of different electrical scheme categories and compress the angular distribution of similar samples during optimization. Experiments show that by using the ArcFace loss function, the model can more accurately distinguish between various electrical scheme images, significantly reducing category confusion, and demonstrating a marked improvement in recognition accuracy on the validation set compared to using the ordinary Softmax function.

[0065] More specifically, suppose the feature vector extracted before the last fully connected layer of ResNet / DINO is... The weight vector for a certain category is ,but and The angle between :

[0066] In the formula, T represents the matrix transpose operation.

[0067] For the true category That term, the ArcFace loss function will replace the corresponding term in the original Softmax function. Replace with ,in, Preset angle intervals (in radians). Other non-realistic categories. of Remain unchanged. Subsequent calculations are based on the adjusted... Cross-entropy loss To update the model parameters.

[0068]

[0069] Where N is the total number of samples; s is the scaling factor; This represents the feature vector of the i-th sample and its true class. The angle between the weight vectors; This indicates the relationship between the sample's feature vector and its category. The angle between the weight vectors; m represents the additional angular interval hyperparameter.

[0070] By adding an angular interval m to the true category items, the model is forced to increase the distance between feature vectors of different electrical scheme categories and compress the angular distribution of feature vectors of the same category during training, thereby improving classification and discrimination performance.

[0071] To further enhance the comprehensive ability of electrical scheme recognition and understanding, this invention also introduces the Qwen-VL-7B multimodal large model for cross-modal feature alignment and fine-tuning of electrical scheme images. This part aims to enable the model not only to "understand" the structure of electrical images, but also to achieve semantic understanding and cross-modal retrieval of electrical schemes in conjunction with textual descriptions. Specifically: The multimodal model consists of two main branches: (1) Vision Encoder: A pre-trained visual Transformer (such as ViT-L / 14 or EVA-CLIP visual backbone) is used to extract features from electrical images to obtain image embedding vectors; (2) Text Encoder: Employs a Transformer-based language model (such as BERT or LLaMA variants) to transform electrical scheme description text into semantic embeddings.

[0072] The two are aligned through a shared feature space, meaning that in the same embedding space, related image and text features are closer together, while irrelevant samples are farther apart. This design enables the model to perform cross-modal tasks such as "image-to-text search" and "text-to-image search," laying the foundation for subsequent automatic annotation, electrical component recognition, and intelligent question answering.

[0073] In practical applications, a large dataset of paired "electrical scheme image-text description" samples can be constructed. Each dataset includes: an electrical scheme diagram and its corresponding list of components and connection descriptions. By cleaning and standardizing these samples, the consistency of the image-text pairings is ensured. Furthermore, to improve robustness, data augmentation strategies can be incorporated, such as image rotation, scaling, grayscale changes, noise perturbation, and paraphrasing and sentence structure variations.

[0074] The prepared "image-text description" pairing data for electrical schemes is input into a TRL or LLAMA Factory framework to perform LoRA fine-tuning on the pre-trained multimodal large model Qwen-VL-7B. In LoRA fine-tuning, gradients are only used to update the low-rank adapter weights injected into the model (such as the attention layer or feedforward network layer of a Transformer), while the main parameters of the pre-trained large model remain frozen. This parameter-efficient fine-tuning (PEFT) strategy allows the model to quickly absorb expertise in the electrical scheme domain and learn accurate image-text correspondences while significantly reducing computational resources and GPU memory usage. As training progresses, the model's retrieval accuracy and the similarity of generated text (such as ROUGE (Recall-Oriented Understudy for Gisting Evaluation) or BLEU (Bilingual Evaluation Understudy) scores on the validation set will steadily improve until convergence.

[0075] Reasoning stage During the system reasoning phase, this invention employs the pre-trained two-stage model for collaborative recognition. The specific process is as follows: 1. Batch Image Detection and Cropping: First, the entire circuit diagram image is processed using a trained YOLO object detection model, outputting the bounding box coordinates of all detected electrical scheme targets. Then, the system crops individual electrical scheme image segments from the original image based on each bounding box. In this way, a circuit image containing multiple electrical schemes is decomposed into several electrical scheme images.

[0076] 2. Electrical Scheme Image Feature Extraction: For each cropped electrical scheme image, a pre-trained image recognition model (ResNet or DINO) is input for forward inference to extract the image's feature vector. The model, trained with ArcFace loss, maps the input electrical scheme image to a... A unit eigenvector (normalized on a unit sphere). Different images of the same electrical scheme category are shown in this figure. In the feature space, they will cluster in close proximity, forming a cluster.

[0077] 3. Generation of Image Index Features for Known Categories: Before system deployment, for each known electrical scheme category, a small number of typical sample images of that category are pre-collected, and their respective feature vectors are extracted using a trained image encoder. The average of these vectors is then calculated to obtain the cluster center vector of that category in the feature space, which serves as the index feature vector for that category. This vector represents the "feature fingerprint" of the electrical scheme in that category. In practice, we normalize the cluster centers of each category to ensure that each vector is of unit length when calculating cosine similarity subsequently.

[0078] 4. Image Similarity Calculation and Classification Decision: For any electrical scheme image to be identified input during the inference stage, after obtaining its feature vector, the system calculates the cosine similarity between it and the index feature vector of each known electrical scheme category in the database. The cosine similarity is defined as:

[0079] in, Let represent the feature vector of the q-th image to be tested. Let represent the index vector of class c. k represents the current dimension, and D represents the total number of dimensions of the vector. Since... and Both are unit vectors, and the above formula calculates the cosine of the angle between them, with a value ranging from [-1, 1]. The closer the similarity is to 1, the closer the angle between the image to be tested and the feature angle of that category (the smaller the angle), and the more likely it is to belong to that category; a similarity close to 0 or negative indicates a very low correlation and a large difference.

[0080] After calculating the similarity scores between the image to be tested and each of the known categories, the largest score is taken and denoted as . The corresponding category is Next, Similarity threshold Compare: If Then the image to be tested is determined to belong to the known category with the highest similarity. ,if If the similarity between the image and a known category is higher than the threshold, the image is classified as "unknown" and does not belong to any known electrical scheme in the current database. This threshold provides a minimum similarity requirement in open set recognition scenarios. Only when the similarity between the image to be tested and a known category is higher than this threshold will the system classify it into that known category, thereby avoiding misidentification of unfamiliar categories as known categories.

[0081] 5. Image Threshold Setting and Open Set Recognition: Similarity Threshold This is a key parameter affecting the performance of open-set recognition. This invention determines a reasonable threshold through experiments on a validation set, without a fixed, unchanging value. For example, in one embodiment, a batch of validation data is prepared, containing both sample images of known categories and several images of new electrical schemes that do not belong to any known category (as "unknown" class samples). Candidate thresholds are selected within the interval [0, 1] for grid search, and the system's recognition performance is evaluated under different thresholds. Two key indicators are examined: the proportion of known category samples correctly identified (known category recall), and the proportion of unknown category samples incorrectly identified as known categories (unknown class misidentification rate). A threshold that minimizes the unknown class misidentification rate and maximizes the known class recognition rate is chosen as the final parameter. For example, the threshold can be set such that the misidentification rate of unknown category samples in the validation set does not exceed approximately 5%. In other words, at this threshold, most unfamiliar electrical scheme images that do not belong to any category in the training set will not be incorrectly classified as a known category, and thus will be correctly identified as "unknown" by the system. At the same time, the threshold is not too stringent, so as to ensure that the vast majority of images belonging to known electrical schemes can exceed the threshold and be matched to the correct category, avoiding the situation where samples of known categories are misclassified as unknown due to an excessively high threshold.

[0082] By adjusting the validation set, this invention determined a balanced and reasonable threshold for practical deployment. This threshold has a clear physical meaning: it represents the minimum similarity level required for the system to determine if an image belongs to a known electrical scheme category. A lower threshold increases the detection rate of known categories but may lead to some unknown category samples being misclassified as known; a higher threshold, on the other hand, reduces the number of misclassifications of unknown categories but may cause a few samples that truly belong to known categories to be classified as unknown due to insufficient similarity. Through validation set experiments, a trade-off was struck between accuracy and false recognition rate, and the optimal threshold was selected for open-set recognition decisions.

[0083] 6. Electrical Scheme Text Feature Extraction: For each cropped electrical scheme image, input a finely tuned qwen-VL-7B multimodal large-scale model and engage in dialogue with it. Input "This image is an electrical scheme; output a list of components and their connections," and receive the model's output response. Then, use pre-trained textembedding models such as Qwen3-8B-Embedding to extract the feature vector of the image's text description.

[0084] 7. Generation of Image Index Features for Known Categories: Before system deployment, for each known electrical scheme category, typical sample images of that category are pre-collected and input into a finely tuned qwen-VL-7B multimodal large-scale model to obtain the model's textual description of the electrical scheme. Then, pre-trained text embedding models such as Qwen3-8B-Embedding are used to extract the feature vector of the image textual description. Finally, the feature vectors of all categories of electrical schemes are stored in vector databases such as Weaviate.

[0085] 8. Text Similarity Calculation and Classification Decision: During real-time inference, the feature vector (V_query) of the electrical scheme to be identified extracted in step 6 is used as the query vector and submitted to the Weaviate vector database constructed in step 7. The vector database utilizes its built-in efficient approximate nearest neighbor search algorithm (HNSW) to quickly retrieve the K most similar known category feature vectors to V_query from the index. The similarity metric uses cosine similarity. Since feature vectors generated by models such as Qwen3-8B-Embedding are usually normalized, cosine similarity can effectively measure the directional consistency of two vectors in the semantic space. The similarity score ranges from [-1, 1], with scores closer to 1 indicating that the textual descriptions of the two electrical schemes are semantically similar.

[0086] 9. Combining Image and Text Results: To achieve more robust classification results, this invention employs a modality fusion strategy, combining image similarity and text similarity for the final decision. Specifically, the maximum similarity score for each class obtained from the image modality is first calculated... Similarity to each class obtained from text modality Perform normalization and calibration (ensuring the scores of both modalities are on the same dimension or probability scale), then calculate the fusion score:

[0087] in The fusion weights, determined on the validation set through grid search or Bayesian optimization (common values ​​range from 0.4 to 0.8, depending on the reliability of the view / text modality), are ultimately selected. And its corresponding score Sim. If Sim > T (fusion threshold, determined by the validation set), then it is classified as category c; otherwise, it is classified as "unknown".

[0088] 10. Expansion of New Categories: The method of this invention does not require any parameter updates to the original recognition model when introducing new electrical scheme categories; the recognition range can be expanded solely through feature space calculations. When a new electrical scheme brand or category appears, only a very small number (e.g., 3 images) of representative electrical schemes of the new category need to be collected, covering as many common appearance angles and lighting conditions as possible. These new samples are input into a pre-trained image encoder (a ResNet or DINO model fine-tuned by ArcFace) for forward propagation to extract the feature vector of each image. The arithmetic mean of these feature vectors is taken to obtain the cluster center vector of the new category in the feature space, and then normalized. This cluster center vector is then added to the known category index feature list and marked as the new category. Simultaneously, typical sample images of this category are input into a fine-tuned qwen-VL-7B multimodal large model to obtain the large model's textual description of the electrical scheme, and the feature vector of the textual description of the image is extracted using a pre-trained text embedding model such as Qwen3-8B-Embedding. Then, the feature vectors of all electrical scheme categories are stored in vector databases such as Weaviate. This process does not require any backpropagation updates or model retraining; it relies solely on the feature extraction capabilities of the trained model and simple vector operations. The system can flexibly expand to any number of new electrical scheme categories while maintaining the original recognition performance, achieving rapid category expansion and greatly improving the scalability and maintenance efficiency of large-scale electrical scheme management and retrieval systems.

[0089] Figure 3 An embodiment of a computer device according to the present invention is shown. The computer device may be a server, and includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores static and dynamic information data. The network interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the above-described method embodiment.

[0090] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0091] In addition, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0092] In addition, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0093] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0094] This invention is not limited to the structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this invention is limited only by the appended claims.

Claims

1. A CAD electrical scheme identification method characterized by, The method comprises the following steps: An overall CAD electrical scheme image is acquired, and target detection is performed on the overall CAD electrical scheme image to obtain an image segment of each electrical scheme; An image recognition model is used to recognize the image segment to obtain an image feature vector, and similarity calculation is performed on the obtained image feature vector and an image feature vector of a known electrical scheme category to obtain an image similarity of the image segment corresponding to each electrical scheme category; A multi-modal large model is used to recognize the image segment to obtain a text description feature of the image segment, and a text embedding vector model is used to extract the text description feature to obtain a text description feature vector; similarity calculation is performed on the text description feature vector and a text description feature vector of a known electrical scheme category to obtain a text description similarity of the image segment corresponding to each electrical scheme category; A category fusion score is calculated according to the image similarity and the text description similarity, and category classification is performed on the image segment according to the category fusion score.

2. The CAD electrical scheme recognition method of claim 1, characterized in that, The target detection on the overall CAD electrical scheme image comprises the following steps: A pre-trained YOLO target detection model is used to perform target detection on the overall CAD electrical scheme image to obtain a bounding box coordinate of each electrical scheme; The overall CAD electrical scheme image is cropped according to the obtained bounding box coordinate to obtain an image segment of each electrical scheme.

3. The CAD electrical scheme recognition method according to claim 2, characterized in that, The YOLO target detection model is optimized by using a multi-task loss function, wherein a formula of the multi-task loss function is as follows: ； In the formula, represents a multi-task loss function; represents a bounding box regression loss, which is used to optimize the position coincidence degree of the predicted bounding box and the real bounding box; represents a target confidence loss, which is used to judge whether there is an electrical scheme target in the grid or the candidate box; represents a classification loss, which is used to correctly predict the category of the target; 、 、 respectively represent coefficients for balancing the loss contributions of each part.

4. The CAD electrical scheme recognition method according to claim 1, characterized in that, The image recognition model comprises a ResNet network model or a DINO network model, wherein the ResNet network model and the DINO network model are optimized and trained by using an ArcFace loss function.

5. The CAD electrical scheme recognition method according to claim 1, characterized in that, The multi-modal large model comprises a Qwen-VL-7B multi-modal large model.

6. The CAD electrical scheme recognition method of claim 1, wherein, The text embedding vector model comprises a text embedding model trained by using Qwen3-8B-Embedding.

7. The CAD electrical scheme recognition method according to claim 1, characterized in that, The calculation methods of the image similarity and the text description similarity are both cosine similarity calculation methods.

8. The CAD electrical-scheme recognition method of claim 1, characterized in that, A formula of the category fusion score is as follows: ； wherein, represents the class fusion score; is the maximum image similarity for each electrical scheme class; is the text description similarity for each electrical scheme class; represents the fusion weight determined on the validation set by grid search or Bayesian optimization.

9. The CAD electrical-scheme recognition method according to claim 1, characterized in that, The category classification on the image segment according to the category fusion score comprises the following steps: The calculated category fusion score is compared with a predetermined fusion threshold value, and in a case where the comparison result is that the category fusion score is greater than the fusion threshold value, it is determined that the image segment belongs to an electrical scheme category corresponding to the image similarity and the text description similarity.

10. A CAD electrical scheme identification system characterized by, The method comprises the following steps: A target detection module is configured to acquire an overall CAD electrical scheme image, and perform target detection on the overall CAD electrical scheme image to obtain an image segment of each electrical scheme; An image recognition module is configured to use a pre-configured image recognition model to recognize the image segment to obtain an image feature vector, and perform similarity calculation on the obtained image feature vector and an image feature vector of a known electrical scheme category to obtain an image similarity of the image segment corresponding to each electrical scheme category; The text recognition module is configured to recognize the image segment by using a preconfigured multi-modal large model, obtain a text description feature of the image segment, and extract the text description feature by using a text embedding vector model to obtain a text description feature vector; and perform similarity calculation on the text description feature vector and a text description feature vector of a known electrical scheme category to obtain a text description similarity of the image segment corresponding to each electrical scheme category. The category classification module is configured to calculate a category fusion score according to the image similarity and the text description similarity, and perform category classification on the image segment according to the category fusion score.

Images