A high-precision control cabinet terminal identification method
By employing a two-layer network segmentation method and utilizing the SAM model and HOG+SVM feature classification, the problem of insufficient terminal identification accuracy in electrical cabinets was solved, achieving efficient terminal identification and reducing the amount of manual annotation data, thus meeting the requirements of AR inspection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PIPECHINA SOUTH CHINA CO
- Filing Date
- 2023-10-26
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technologies lack sufficient accuracy in terminal identification within electrical cabinets, requiring extensive manual data annotation, making it difficult to meet the needs of efficient AR inspection.
A two-layer network segmentation method is adopted, which uses the SAM model for image mask segmentation and HOG+SVM feature classification to achieve terminal recognition.
It achieves high-precision terminal identification, reduces the amount of manually labeled data, and improves detection efficiency.
Smart Images
Figure CN117557771B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image recognition, and in particular to a high-precision control cabinet terminal recognition method. Background Technology
[0002] Natural gas stations are the core units of natural gas pipeline systems. In long-distance natural gas pipeline systems, they are merely scattered nodes along the route, but they undertake multiple tasks such as receiving, pressurizing, distributing, pigging, and metering. These crucial functions establish the extremely important position of gas stations within the entire pipeline system. The station control room is the heart of the natural gas station, undertaking a large number of monitoring, control, and maintenance tasks. Therefore, the workload involved in monitoring and maintenance is also more heavy, requiring faster location and resolution of problems.
[0003] AR (Augmented Reality) inspection is a method that uses augmented reality technology to improve and enhance the inspection process. AR technology provides real-time visual and information feedback by overlaying virtual information onto the real world, helping inspection personnel to perform inspection tasks more accurately and efficiently.
[0004] As the control center of a natural gas station, the electrical control cabinet is through which most data flows, making it a crucial data source for monitoring the station's safety status. Routine maintenance and repair work is also typically performed here. To achieve efficient inspection and maintenance using AR technology, it is essential to accurately detect and correctly identify each terminal within the electrical cabinet to enable further functionality.
[0005] There are currently many technological gaps in the identification and positioning of tiny terminals inside electrical control cabinets. Due to the high degree of repetition and the extremely dense distribution of terminals, accurately detecting each terminal is very difficult. Current methods suffer from the following problems:
[0006] (1) Limited detection accuracy: There are many target detection methods available now, and various industries will also customize them for their respective application scenarios. However, most of them are used for the detection of targets of general size. For the detection of densely repeated small targets, the difficulty is quite high.
[0007] (2) A large amount of manually labeled data is required: The target detection model based on supervised training must be trained using a large amount of manually labeled data. However, for this type of target scenario, the workload of labeling data is extremely large and there is very little publicly available labeled data.
[0008] (3) Difficult to implement in practice: Based on the limitations of the first two points and the existing methods, it is difficult to implement a target detection and positioning algorithm for natural gas station control cabinets and electrical cabinets to meet the needs of AR inspection. Summary of the Invention
[0009] The present invention mainly addresses the technical problems of insufficient detection accuracy and the need for a large amount of manual data annotation in existing technologies, and provides a high-precision control cabinet terminal identification method.
[0010] The present invention addresses the aforementioned technical problems primarily through the following technical solution: a high-precision control cabinet terminal identification method, comprising the following steps:
[0011] S1. Capture an image of the control cabinet containing the power strip as the original image;
[0012] When using VR equipment for shooting, the VR equipment should be positioned at an appropriate distance directly in front of the electrical cabinet. Adjust the height so that the object to be identified (wiring terminals, control buttons, or other targets) is directly in front of it. The resolution of the captured image is generally 3904*2196.
[0013] S2. Process the original image using the SAM model to obtain the mask for each region.
[0014] S3. Based on the masking of each region obtained in step S2, segment the original image to obtain several first segmentation images and label them with serial numbers, and convert all first segmentation images to a uniform resolution (600*600).
[0015] S4. Use the HOG feature extraction algorithm to extract HOG features from each of the first segmented images with uniform resolution.
[0016] S5. Input all the HOG features extracted in step S4 into the classification and interleaving SVM model for classification, and obtain the sequence number of the interleaving image in the first segmented image. The first segmented image corresponding to the sequence number is recorded as the original interleaving image. The classification and interleaving SVM model is trained through the interleaving training set, which contains 10 interleaving images as positive examples and other 10 non-interleaving images as negative examples. There may be multiple original interleaving images. If there are multiple images, they will be processed separately in the following steps.
[0017] S6. Use the SAM model to process the original interlaced image and obtain the mask for each region.
[0018] S7. Based on the masking of each region obtained in step S6, the original interleaved image is segmented to obtain several second segmentation images, and all second segmentation images are converted to a uniform resolution (600*600).
[0019] S8. Use the HOG feature extraction algorithm to extract HOG features from each of the second segmented images with uniform resolution.
[0020] S9. Input all the HOG features extracted in step S8 into the classification terminal feature SVM model for classification, and obtain the images belonging to the terminal features in the second segmentation image; the classification terminal feature SVM model is trained with a training set containing 20 positive examples and 20 negative examples;
[0021] S10. Based on the image of the terminal features obtained in step S9, the position of the terminal on the original image is obtained by coordinate transformation.
[0022] Preferably, in step S2, the SAM model includes an image encoder, a cue encoder, and a mask encoder;
[0023] The image encoder maps the image to be processed to the image feature space. This scheme uses a pre-trained VisionTransformer (ViT) as the image encoder.
[0024] The cue encoder maps the input cues to the feature space of the cues. There are two sets of cues: the first set consists of points, boxes, and text (sparse cues), and the second set consists of masks (dense cues). Points and boxes are represented by positional encodings, which combine learned embeddings from each cue with text of any form (processed using CLIP). Masks are summed with the elements of the image after convolutional embedding.
[0025] The mask decoder maps the graph embedding, cue embedding, and output labels to a mask. It adds a dynamic mask prediction head after the Transformer's decoder, mapping the MLP's output labels to a dynamic linear classifier before calculating the mask foreground probability at each image location. The mask decoder modifies prompt-to-image embedding and vice-versa using cue self-attention and cross-attention. After these two parts, the image is upsampled, and the MLP is used again to map the output labels to the dynamic linear classifier, ultimately yielding the mask foreground probability at each image location.
[0026] The SAM model in step S6 has the same structure as the SAM model in step S2.
[0027] Preferably, the HOG feature extraction algorithm specifically involves: converting the image to grayscale and performing gamma correction, then calculating the image pixel gradient map, and finally performing block normalization to obtain the HOG features.
[0028] The HOG feature extraction algorithm can convert a 3-channel color image into a feature vector of a certain length (HOG feature). The main purpose of the HOG feature extraction algorithm is to calculate the gradient of the image and statistically analyze the gradient direction and magnitude. The extracted edge and gradient features effectively capture the characteristics of local shapes, and because the image undergoes Gamma correction and cell-based normalization, it exhibits good invariance to geometric and optical changes; transformations or rotations have minimal impact on sufficiently small regions.
[0029] Preferably, the parameters of the SAM model in step S2 are as follows:
[0030] The SAM sampling point density (points_per_side) is 13; the higher the density, the more masks are obtained. The IOU threshold (pred_iou_thresh) is 0.82, and the IOU threshold ranges from [0,1]. The larger this value is, the closer the bounding box used by the labeled mask is to the training set. The confidence threshold (stability_score_thresh) is 0.92, and the confidence threshold ranges from [0,1]. The larger this value is, the higher the confidence of the labeled mask. The minimum mask area (min_mask_region_area) is 1000; the smaller the minimum mask area, the smaller the mask obtained.
[0031] Preferably, the parameters of the SAM model in step S6 are as follows:
[0032] The SAM sampling point density is 18, the IOU threshold is 0.82, the confidence threshold is 0.92, and the minimum masking area is 1000.
[0033] Preferably, in step S9, the terminal feature identified by the SVM model is the dot directly above the terminal.
[0034] Considering that SAM has a low success rate when processing the entire terminal directly, and SVM also performs poorly in classifying the entire terminal, this invention selects the dotted area above the terminal's numerical label as the recognition region and the SVM dataset. Each terminal has a dot directly above it, and the relative position of the dot and the terminal is fixed. Therefore, the terminal's position can be obtained directly from the dot's position by coordinate transformation (shifting the vertical axis down a certain position), thus obtaining the terminal image and providing an image basis for subsequent terminal state detection.
[0035] The goal of SVM is to find an optimal hyperplane (or a hyperplane in high-dimensional space) that separates samples of different classes and maximizes the margin between the two classes. This invention trains an SVM model for classifying interleaved samples (applied to step S5) and an SVM model for classifying terminal samples (applied to step S9) using images captured by a camera.
[0036] To address the limitations of traditional YOLO models in terms of detection accuracy and the large amount of labeled data required, this technique employs a two-layer network segmentation-based target detection method using the SAM model and HOG+SVM approach, achieving high-precision identification of control cabinet terminals. Specifically, the SAM model is used to process the original image and the connector image (adding masks), while the two SVM models are used to perform HOG feature classification on the connector image and the terminal image, respectively.
[0037] The substantial effect of this invention is that it achieves higher-precision target detection, eliminating the huge workload and tedious engineering of manual data annotation. Instead, it uses the results of two-layer semantic segmentation as annotation data for feature classification supervision. Attached Figure Description
[0038] Figure 1 This is a schematic diagram of a process of the present invention;
[0039] Figure 2 This is an original image of the present invention;
[0040] Figure 3 This is an image processed by SAM and labeled with a mask according to the present invention;
[0041] Figure 4 This is an original image of a power strip according to the present invention;
[0042] Figure 5 This invention provides an interleaved image after SAM processing and mask annotation.
[0043] Figure 6 This is an image of a dot above a terminal according to the present invention;
[0044] Figure 7 This is an illustrated image of a power strip as described in this invention.
[0045] Figure 8 This is a model structure diagram of the present invention. Detailed Implementation
[0046] The technical solution of the present invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings.
[0047] Example: This example illustrates a high-precision control cabinet terminal identification method, such as... Figure 1 As shown, it includes the following steps:
[0048] S1. Capture an image of the control cabinet, including the power strip, as the original image, such as... Figure 2 As shown;
[0049] When using VR equipment for shooting, the VR equipment should be positioned at an appropriate distance directly in front of the electrical cabinet. Adjust the height so that the object to be identified (wiring terminals, control buttons, or other targets) is directly in front of it. The resolution of the captured image is generally 3904*2196.
[0050] S2. Process the original image using the SAM model to obtain the masking for each region. Figure 3 The image is after SAM processing and mask annotation;
[0051] S3. Based on the masking of each region obtained in step S2, segment the original image to obtain several first segmentation images and label them with serial numbers, and convert all first segmentation images to a uniform resolution (600*600).
[0052] S4. Use the HOG feature extraction algorithm to extract HOG features from each of the first segmented images with uniform resolution.
[0053] S5. Input all the HOG features extracted in step S4 into the classification and interleaving SVM model for classification, and obtain the index of the interleaving image in the first segmented image. The first segmented image corresponding to the index is recorded as the original interleaving image, such as... Figure 4 As shown; the classification and arrangement SVM model is trained using an arrangement training set, which contains 10 arrangement images as positive examples and 10 images without arrangement as negative examples; there may be multiple original arrangement images obtained, and each image will be processed separately in later steps.
[0054] S6. Process the original interlaced image using the SAM model to obtain the masking of each region, such as... Figure 5 As shown;
[0055] S7. Based on the masking of each region obtained in step S6, the original interleaved image is segmented to obtain several second segmentation images, and all second segmentation images are converted to a uniform resolution (600*600).
[0056] S8. Use the HOG feature extraction algorithm to extract HOG features from each of the second segmented images with uniform resolution.
[0057] S9. Input all the HOG features extracted in step S8 into the terminal feature SVM model for classification, and obtain the images belonging to the terminal features in the second segmented image, such as... Figure 6 As shown; the classification terminal feature SVM model is trained using a training set containing 20 positive examples and 20 negative examples;
[0058] S10. Based on the image of the terminal features obtained in step S9, the position of the terminal on the original image is obtained by coordinate transformation.
[0059] Figure 8 This is a model structure diagram of this embodiment.
[0060] In step S2, the SAM model includes an image encoder, a cue encoder, and a mask encoder.
[0061] The image encoder maps the image to be processed to the image feature space. This scheme uses a pre-trained VisionTransformer (ViT) as the image encoder.
[0062] The cue encoder maps the input cues to the feature space of the cues. There are two sets of cues: the first set consists of points, boxes, and text (sparse cues), and the second set consists of masks (dense cues). Points and boxes are represented by positional encodings, which combine learned embeddings from each cue with text of any form (processed using CLIP). Masks are summed with the elements of the image after convolutional embedding.
[0063] The mask decoder maps the graph embedding, cue embedding, and output labels to a mask. It adds a dynamic mask prediction head after the Transformer's decoder, mapping the MLP's output labels to a dynamic linear classifier before calculating the mask foreground probability at each image location. The mask decoder modifies prompt-to-image embedding and vice-versa using cue self-attention and cross-attention. After these two parts, the image is upsampled, and the MLP is used again to map the output labels to the dynamic linear classifier, ultimately yielding the mask foreground probability at each image location.
[0064] The SAM model in step S6 has the same structure as the SAM model in step S2.
[0065] The HOG feature extraction algorithm specifically involves: converting the image to grayscale and performing gamma correction, then calculating the image pixel gradient map, and finally performing block normalization to obtain the HOG features.
[0066] The HOG feature extraction algorithm can convert a 3-channel color image into a feature vector of a certain length (HOG feature). The main purpose of the HOG feature extraction algorithm is to calculate the gradient of the image and statistically analyze the gradient direction and magnitude. The extracted edge and gradient features effectively capture the characteristics of local shapes, and because the image undergoes Gamma correction and cell-based normalization, it exhibits good invariance to geometric and optical changes; transformations or rotations have minimal impact on sufficiently small regions.
[0067] The parameters of the SAM model in step S2 are as follows:
[0068] The SAM sampling point density (points_per_side) is 13; the higher the density, the more masks are obtained. The IOU threshold (pred_iou_thresh) is 0.82, and the IOU threshold ranges from [0,1]. The larger this value is, the closer the bounding box used by the labeled mask is to the training set. The confidence threshold (stability_score_thresh) is 0.92, and the confidence threshold ranges from [0,1]. The larger this value is, the higher the confidence of the labeled mask. The minimum mask area (min_mask_region_area) is 1000; the smaller the minimum mask area, the smaller the mask obtained.
[0069] The parameters of the SAM model in step S6 are as follows:
[0070] The SAM sampling point density is 18, the IOU threshold is 0.82, the confidence threshold is 0.92, and the minimum masking area is 1000.
[0071] In step S9, the terminal feature identified by the SVM model is the dot directly above the terminal.
[0072] Considering that SAM has a low success rate in processing the entire terminal and SVM also performs poorly in classifying the entire terminal, this invention selects the dotted portion above the terminal's numerical label as the recognition area and the SVM dataset. For example... Figure 7 The image shown is an image of the connector after being identified by the algorithm. Each terminal has a dot directly above it. The relative position of the dot and the terminal is fixed. Therefore, the position of the terminal can be obtained by directly transforming the coordinates of the dot (shifting the vertical axis down by a certain position), thus obtaining the terminal image and providing an image basis for subsequent terminal status detection.
[0073] The goal of SVM is to find an optimal hyperplane (or a hyperplane in high-dimensional space) that separates samples of different classes and maximizes the margin between the two classes. This invention trains an SVM model for classifying interleaved samples (applied to step S5) and an SVM model for classifying terminal samples (applied to step S9) using images captured by a camera.
[0074] Because power strips and control buttons in electrical cabinets are generally neatly arranged in horizontal or vertical rows, an entire row or column is relatively large and easy to identify and detect. However, each terminal is very small and generally difficult to directly detect and segment. Yet, each terminal is a relatively large target within its row or column, making it easier to accurately detect and segment. Therefore, this solution constructs a two-layer semantic segmentation model, called a coarse segmentation model and a fine segmentation model. The coarse segmentation model detects entire rows of power strips, identifying all terminals within the entire strip as a single target and segmenting them. The fine segmentation model, using the output of the coarse segmentation model as input, finely segments the entire row of power strips, accurately segmenting each terminal.
[0075] For each precisely segmented target, HOG is used to extract its robust features, and support vector machine (SVM) is used to classify and identify the features, ultimately obtaining the category attributes of the precisely segmented target.
[0076] This solution abandons the traditional YOLO model-based object detection method and instead adopts the SAM model to achieve object detection through multiple semantic segmentations in a predetermined manner. High-precision semantic segmentation is computationally expensive and unsustainable, while low-precision semantic segmentation cannot segment high-precision targets. Therefore, we employ a two-level segmentation approach. The first segmentation extracts targets at the connector level, and a second segmentation is performed based on this connector level to obtain connector-level segments. After obtaining the segmented small targets through the two-level segmentation, the HOG+SVM approach is used to classify and recognize the targets, completing the final recognition task and achieving the target detection and recognition of the desired target.
[0077] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.
[0078] Although this paper uses terms such as SAM model, HOG feature, and masking extensively, the possibility of using other terms is not excluded. These terms are used merely for the convenience of describing and explaining the essence of this invention; interpreting them as any additional limitation would contradict the spirit of this invention.
Claims
1. A high-precision control cabinet terminal identification method, characterized in that, Includes the following steps: S1. Capture an image of the control cabinet containing the power strip as the original image; S2. Process the original image using the SAM model to obtain the masking of each region; S3. Based on the masking of each region obtained in step S2, segment the original image to obtain several first-segmentation images and label them with serial numbers, and convert all first-segmentation images into a uniform resolution; S4. Use the HOG feature extraction algorithm to extract HOG features from each of the first segmented images with uniform resolution. S5. Input all the HOG features extracted in step S4 into the classification and interleaving SVM model for classification, and obtain the sequence number of the interleaving image in the first segmented image. The first segmented image corresponding to the sequence number is recorded as the original interleaving image. S6. Use the SAM model to process the original interlaced image and obtain the masking of each region; S7. Based on the masking of each region obtained in step S6, the original interleaved image is segmented to obtain several second segmented images, and all second segmented images are converted to a uniform resolution. S8. Use the HOG feature extraction algorithm to extract HOG features from each of the second segmented images with uniform resolution. S9. Input all the HOG features extracted in step S8 into the classification terminal feature SVM model for classification, and obtain the images belonging to the terminal features in the second segmentation image; S10. Based on the image of the terminal features obtained in step S9, the position of the terminal on the original image is obtained by coordinate transformation.
2. The high-precision control cabinet terminal identification method according to claim 1, characterized in that, In step S2, the SAM model includes an image encoder, a cue encoder, and a mask encoder. The image encoder maps the image to be processed to the image feature space. The cue encoder maps the input cue to the cue feature space. The cue has two sets: the first set consists of points, boxes, and text, and the second set consists of a mask. The points and boxes are represented by positional encoding, and the mask is summed with the image elements after convolutional embedding. The mask decoder maps the image embedding, cue embedding, and output label to the mask. The SAM model in step S6 has the same structure as the SAM model in step S2.
3. A high-precision control cabinet terminal identification method according to claim 1 or 2, characterized in that, The HOG feature extraction algorithm specifically involves: converting the image to grayscale and performing gamma correction, then calculating the image pixel gradient map, and finally performing block normalization to obtain the HOG features.
4. The high-precision control cabinet terminal identification method according to claim 1, characterized in that, The parameters of the SAM model in step S2 are as follows: The SAM sampling point density is 13, the IOU threshold is 0.82, the confidence threshold is 0.92, and the minimum masking area is 1000.
5. The high-precision control cabinet terminal identification method according to claim 1, characterized in that, The parameters of the SAM model in step S6 are as follows: The SAM sampling point density is 18, the IOU threshold is 0.82, the confidence threshold is 0.92, and the minimum masking area is 1000.
6. A high-precision control cabinet terminal identification method according to claim 1 or 5, characterized in that, In step S9, the terminal feature identified by the SVM model is the dot directly above the terminal.