Method and device for identifying wiring state of terminal wire of screen cabinet based on semantic segmentation

Abstract

The invention discloses a method and device for recognizing the wiring state of a screen cabinet terminal wire based on semantic segmentation, and relates to the field of electrical equipment state detection. In the method, a visible light image, an infrared thermal image and a depth image are fused to obtain a fused image; obtaining a fused image of the region of interest of the terminal block from the fused image; semantic segmentation and context analysis are carried out on the terminal strip region-of-interest fusion image to obtain a pixel segmentation region of a target terminal port, similarity matching is carried out according to terminal line features in the pixel segmentation region and samples in a preset database to determine a judgment threshold value, and the wiring on-off state of the target terminal port is determined according to the judgment threshold value; and obtaining the texture features and color features of the sleeve with the terminal port and the relative position relationship with the target terminal line, and determining the installation state of the sleeve with the terminal port. By implementing the technical scheme provided by the invention, the wiring state of the terminal wire of the screen cabinet and the installation state of the sleeve can be accurately and efficiently identified.

Description

Technical Field

[0001] The present application relates to the technical field of electrical equipment status detection, and specifically to a method and device for identifying the wiring status of panel cabinet terminal wires based on semantic segmentation. Background Art

[0002] In modern power systems, cabinets, as an important component of electrical equipment, play a key role in collecting and distributing electrical energy, controlling circuit switching, and protecting electrical equipment. As power systems continue to develop and become more complex, cabinets are also increasing in size and complexity. The wiring status of terminal cables within cabinets is directly related to the safe and stable operation of the power system. Accurately identifying the wiring status of cabinet terminal cables is crucial to ensuring power system reliability.

[0003] In existing technology, identifying the wiring status of cabinet terminal wires often relies on manual inspections. Workers visually inspect the wiring inside the cabinet to check for secure connections and any loose or falling wires. Other methods rely on single-type images. For example, these methods use a visible light camera to capture an image of the cabinet's exterior and determine the wiring status based on features like the shape and color of the terminal wires. While this method can identify wiring status to a certain extent, it can be difficult to accurately detect hidden wiring issues.

[0004] However, these existing technologies have significant drawbacks. Manual inspections are inefficient and unable to meet the frequent inspection needs of large-scale, complex cabinets. Furthermore, in some confined or dangerous spaces, manual inspections are difficult to perform and may even pose safety risks. Recognition methods based on single-type images, due to their limited information dimensions, struggle to fully and accurately identify wiring status, especially for wiring issues that may be difficult to detect but pose hidden risks. Summary of the Invention

[0005] The present application provides a method and device for identifying the wiring status of panel cabinet terminal wires based on semantic segmentation, which can accurately and efficiently identify the wiring status of panel cabinet terminal wires and the installation status of bushings, thereby ensuring the safe and stable operation of electrical equipment.

[0006] In a first aspect of the present application, a method for identifying the wiring status of a panel cabinet terminal line based on semantic segmentation is provided, which is applied to a panel cabinet terminal line wiring status identification platform. The method comprises: Use a visible light camera, an infrared camera, and a depth camera to simultaneously shoot the screen cabinet to obtain a visible light image, an infrared thermal image, and a depth image, and fuse the visible light image, the infrared thermal image, and the depth image to obtain a fused image; Acquiring target features from the fused image, matching the target features with preset terminal strip template features to obtain a preliminary positioning result, and processing the preliminary positioning result using geometric constraints to obtain a fused image of a terminal strip region of interest; Performing semantic segmentation and context analysis on the fused image of the terminal block region of interest to obtain a pixel segmentation region of the target terminal port, acquiring terminal line features from the pixel segmentation region, performing similarity matching on the terminal line features with samples in a preset database to determine a judgment threshold, and determining the connection and disconnection status of the target terminal port based on the judgment threshold; The texture features, color features, and relative positional relationship between the sleeve of the connected terminal port and the target terminal wire are obtained to determine the installation state of the sleeve of the connected terminal port, where the target terminal wire is the terminal wire connected to the sleeve.

[0007] By employing this technical solution, multimodal images are captured and fused using visible light, infrared, and depth cameras. This integrated image information from these different types provides a more comprehensive and accurate representation of the terminal block and its wiring conditions, overcoming the information acquisition limitations of a single image mode and improving the reliability of subsequent feature extraction and analysis. Feature matching and geometric constraint processing allow precise localization of the terminal block's region of interest within the fused image, reducing interference from irrelevant areas. This provides a foundation for accurate analysis of target areas such as terminal ports, improving overall recognition efficiency and accuracy. Semantic segmentation and contextual analysis are performed on the fused image of the terminal block's region of interest to precisely determine the pixel-wise segmentation of the target terminal port. This, by considering the spatial and visual relationships between the target terminal port and adjacent ports, effectively improves the accuracy of terminal port recognition and avoids false or missed identification. Terminal line features are extracted from the pixel-wise segmentation of the target terminal port and compared with a pre-set database of samples to determine a threshold for determining the connection status. This method, based on a large amount of sample data and similarity comparison, objectively and accurately determines the connection status of the wiring, reducing the subjectivity and error inherent in human judgment. Obtain the texture characteristics, color characteristics and relative position relationship of the bushing of the wired terminal port and the target terminal line, comprehensively determine the installation status of the bushing based on multiple information, comprehensively evaluate the compliance and quality of the bushing installation, promptly detect any abnormalities in the bushing installation, and ensure the overall quality and stability of the terminal line wiring in the panel cabinet.

[0008] Optionally, fusing the visible light image, the infrared thermal image, and the depth image to obtain a fused image includes: Aligning the visible light image, the infrared thermal image, and the depth image based on a feature point registration method; Acquiring visual features of the visible light image, thermal distribution features of the infrared thermal image, and geometric features of the depth image; The visual features, the thermal distribution features, and the geometric features are spliced ​​to form a fusion feature vector, and the fusion image is obtained according to the fusion feature vector.

[0009] By adopting the above technical solution, visible light, infrared thermal, and depth images are aligned through a feature point-based registration method, achieving precise integration of different types of image information. This allows the fused image to combine the advantages of multiple image modes and provide richer and more comprehensive information. The visual features of the visible light image, the thermal distribution features of the infrared thermal image, and the geometric features of the depth image are separately acquired and spliced ​​together to form a fused feature vector. This fully utilizes the unique feature information of each image type and enhances the fused image's ability to describe the target scene. The fused image obtained based on the fused feature vector incorporates multiple image features and, compared to a single image, has significantly improved information content, accuracy, and clarity, facilitating more accurate analysis and identification of subsequent targets (such as terminal blocks in panel cabinets).

[0010] Optionally, the feature point-based registration method for aligning the visible light image, the infrared thermal image, and the depth image includes: Acquire a first key point and a first feature descriptor of the visible light image, a second key point and a second feature descriptor of the infrared thermal image, and a third key point and a third feature descriptor of the depth image; Performing a first matching operation on the first feature descriptor and the second feature descriptor to obtain a first matching pair, performing a second matching operation on the first feature descriptor and the third feature descriptor to obtain a second matching pair, calculating a first transformation matrix from the infrared thermal image to the visible light image based on the first matching pair, and calculating a second transformation matrix from the depth image to the visible light image based on the second matching pair; The infrared thermal image is converted into the coordinate system of the visible light image by applying the first transformation matrix, the depth image is converted into the coordinate system of the visible light image by applying the second transformation matrix, and pixel values ​​of the converted images are interpolated using an interpolation method.

[0011] By adopting the above technical solution, the key points and feature descriptors of visible light, infrared thermal and depth images are obtained respectively, and feature matching is performed to obtain matching pairs. The corresponding feature points in different images can be accurately found, providing a reliable basis for subsequent coordinate transformation and ensuring the accuracy of image alignment. Based on the matching pairs, the transformation matrix from infrared thermal image to visible light image and depth image to visible light image is calculated to realize the transformation of different images to a unified coordinate system, effectively solving the problem of inconsistent spatial positions of multimodal images and enabling images to be fused and analyzed in the same coordinate system. The pixel values ​​of the converted images are interpolated using an interpolation method to compensate for the pixel loss or distortion problems that may occur during the image transformation process, improve the continuity and clarity of the image, and provide high-quality image data for subsequent processing and analysis tasks based on fused images.

[0012] Optionally, performing semantic segmentation and context analysis on the fused image of the terminal block region of interest to obtain the pixel segmentation region of the target terminal port includes: Using a semantic segmentation algorithm to classify the target pixel in the fused image of the terminal strip region of interest, and determine whether the target pixel belongs to the target terminal opening region, so as to obtain a preliminary semantic segmentation result, wherein the target pixel is any pixel in the fused image of the terminal strip region of interest; Analyzing the distance and relative position relationship between a target terminal port and adjacent terminal ports, as well as the brightness and contrast of the region where the target terminal port is located, to adjust the preliminary semantic segmentation result, wherein the target terminal port is any terminal port in the preliminary semantic segmentation result; The pixel segmentation area of ​​the target terminal port is constructed according to the adjusted preliminary semantic segmentation result.

[0013] By adopting the above technical solution, the target pixels in the fused image of the terminal block area of ​​interest are classified using a semantic segmentation algorithm. This allows for a preliminary and accurate determination of whether the target pixel belongs to the target terminal port area, resulting in a preliminary semantic segmentation result. This provides a basis for subsequent processing and effectively distinguishes the target terminal port area from other areas. The preliminary semantic segmentation result is adjusted by analyzing contextual information such as the distance and relative positional relationship between the target terminal port and adjacent terminal ports, as well as the brightness and contrast of the area in which it is located. This fully utilizes the spatial and visual relationship between the target terminal port and its surroundings in the image, overcomes the errors that may occur when relying solely on pixel feature segmentation, and improves the accuracy of target terminal port segmentation. Constructing the pixel segmentation area of ​​the target terminal port based on the adjusted preliminary semantic segmentation result can obtain an accurate pixel segmentation area of ​​the target terminal port, providing an accurate data basis for subsequent further analysis of the target terminal port (such as wiring status judgment), and helping to improve the performance of the entire cabinet terminal line wiring status recognition system.

[0014] Optionally, performing similarity matching between the terminal wire feature and samples in a preset database to determine a determination threshold, and determining the connection status of the target terminal port according to the determination threshold includes: Comparing and calculating the terminal wire features with each sample feature in a preset database one by one to obtain a similarity value set; Dividing the similarity values ​​in the similarity value set into a first similarity value set and a second similarity value set according to the connection on / off state labels; determining a determination threshold according to a first distribution characteristic of similarity values ​​in the first similarity value set and a second distribution characteristic of similarity values ​​in the second similarity value set; When the average similarity value in the target similarity value set in the similarity set is greater than or equal to the judgment threshold, the wiring of the target terminal port is determined to be in an on state; otherwise, the wiring of the target terminal port is determined to be in a disconnected state. The target similarity value set is a set of similarity values ​​whose similarity values ​​are higher than a preset threshold.

[0015] By employing this technical solution, the terminal wire features are individually compared with each sample feature in a pre-set database to generate a set of similarity values. This quantitatively assesses the degree of similarity between the terminal wire features and the sample features, providing a concrete data basis for subsequent wiring status determination and avoiding errors caused by subjective judgment. The similarity value set is divided into two subsets based on the wiring connection / disconnection status label, and a threshold is determined based on the distribution characteristics of the similarity values ​​in the two subsets. This threshold determination method based on the distribution of actual sample data fully accounts for the differences in feature similarity under different wiring states, making the threshold more scientific and reasonable, and effectively distinguishing between the connected and disconnected states. Using the determined threshold as the standard, the target terminal port's connection / disconnection status is accurately determined by comparing the average similarity value of the target similarity value set (the set with similarity values ​​above the pre-set threshold) with the threshold. This judgment method, based on objective data comparison, improves the accuracy and reliability of wiring status determination, helping to promptly detect wiring faults and ensure the normal operation of the system.

[0016] Optionally, acquiring the texture features, color features, and relative positional relationship between the sleeve of the connected terminal port and the terminal wire to determine the installation state of the sleeve of the connected terminal port includes: The target area image of the casing is processed using a local binary pattern algorithm to calculate the LBP value of each pixel, and the texture feature vector of the casing is determined according to the total LBP value of the target area image; Converting the target area image from the RGB color space to the HSV color space, respectively calculating the color histograms of the target area image in three channels, namely, hue, saturation, and lightness, and concatenating the color histograms of the three channels to form a color feature vector of the sleeve; Calculating the Euclidean distance between the center point of the sleeve and the center point of the target terminal line, and the vertical distance from the center point of the sleeve to the straight line on which the target terminal line is located, and determining the relative positional relationship between the sleeve and the target terminal line according to the Euclidean distance and the vertical distance; If the texture feature vector, the color feature vector, and the relative position relationship are all within corresponding preset ranges, it is determined that the installation state of the sleeve is normal; otherwise, it is determined that the installation state of the sleeve is abnormal.

[0017] By employing the above technical solution, the local binary pattern algorithm is used to extract the casing texture feature vector. The image is converted from RGB to HSV color space, and the color histogram of each channel is statistically analyzed to form a color feature vector. The relative position relationship between the casing and the terminal wire is calculated. This method comprehensively obtains information related to the casing installation status from three dimensions: texture, color, and spatial position, providing a rich basis for accurate judgment. Based on the extracted texture feature vector, color feature vector, and relative position relationship, they are compared with the corresponding preset ranges. The normal installation status of the casing is determined by integrating multiple factors. This method can effectively avoid the errors that may be caused by single-factor judgments and improve the accuracy and reliability of casing installation status judgment. The method can quickly detect the casing installation status. If any feature of texture, color, or relative position relationship is found to be outside the preset range, it can be determined as an abnormal installation state. This helps to promptly identify casing installation problems, facilitate timely repair or adjustment measures, and ensure the overall quality and stability of the cabinet terminal wiring.

[0018] Optionally, the method further includes: A knowledge graph of the wiring status of the terminal wires in the panel cabinet is constructed, using the terminal port, terminal wire, and bushing as entity nodes, and the wiring relationship and bushing installation relationship as the relationship edges between the entities. For terminal ports whose wiring on / off status does not conform to the wiring specifications in the knowledge graph, mark them and extract correction suggestions from the knowledge graph; For the terminal ports that have been wired, a matching analysis is performed based on the installation status of the bushing and the fault mode library in the knowledge graph to predict potential failure risks, and corresponding early warning information is given based on the potential failure risks.

[0019] By adopting the above technical solution, a knowledge graph of the wiring status of the terminal wires in the panel cabinet is constructed, and entities such as terminal ports, terminal wires, bushings, and their wiring relationships, bushing installation relationships, etc. are presented in a structured manner, realizing the systematic integration of relevant knowledge, which is convenient for subsequent rapid query and reasoning analysis using knowledge. Terminal ports whose wiring on / off status does not conform to the wiring specifications in the knowledge graph are marked, and correction suggestions are automatically extracted, which can quickly locate abnormal situations and provide solutions, improve the efficiency and accuracy of fault handling, and reduce the time cost of manual troubleshooting and decision-making. Based on the fault mode library in the knowledge graph, the bushing installation status of the connected terminal ports is matched and analyzed, and potential fault risks are predicted and early warning information is given, realizing the transition from passive fault handling to active risk prevention, which helps to take measures in advance to avoid faults and ensure the stability and reliability of the terminal wire wiring in the panel cabinet.

[0020] In a second aspect of the present application, a system for identifying the connection status of terminal wires in a panel cabinet based on semantic segmentation is provided, comprising an acquisition module, a positioning module, an analysis module, and an identification module, wherein: an acquisition module configured to simultaneously photograph the screen cabinet using a visible light camera, an infrared camera, and a depth camera to obtain a visible light image, an infrared thermal image, and a depth image, and fuse the visible light image, the infrared thermal image, and the depth image to obtain a fused image; a positioning module configured to obtain target features from the fused image, match the target features with preset terminal strip template features to obtain a preliminary positioning result, and process the preliminary positioning result using geometric constraints to obtain a fused image of the terminal strip region of interest; an analysis module configured to perform semantic segmentation and context analysis on the fused image of the terminal block region of interest to obtain a pixel segmentation region of a target terminal port, obtain terminal line features from the pixel segmentation region, perform similarity matching on the terminal line features with samples in a preset database to determine a determination threshold, and determine the connection / disconnection state of the target terminal port according to the determination threshold; The identification module is configured to obtain the texture characteristics, color characteristics and relative position relationship of the sleeve of the connected terminal port and the target terminal wire, and determine the installation status of the sleeve of the connected terminal port, and the target terminal wire is the terminal wire connected to the sleeve.

[0021] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device performs any of the methods described above.

[0022] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions. When the instructions are executed, any one of the methods described above is executed.

[0023] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By acquiring and fusing multimodal images using visible light, infrared, and depth cameras, and integrating multiple image information, it can more comprehensively and accurately present the terminal blocks and wiring conditions of the panel cabinets, providing a rich and accurate data foundation for subsequent analysis and reducing information loss or errors caused by a single image mode. 2. Through feature matching and geometric constraint processing, the terminal strip area of ​​interest is accurately located from the fused image, effectively focusing on the key area, eliminating irrelevant interference, and improving the efficiency and accuracy of subsequent analysis of target areas such as terminal ports; 3. Perform semantic segmentation and context analysis on the fused image of the terminal strip's region of interest to obtain pixel segmentation of the target terminal port. Terminal line features are extracted and similarity matched against samples from a preset database to determine the judgment threshold, thereby determining the connection status. This method, based on multi-dimensional feature analysis and similarity comparison, can objectively and accurately determine the connection status and reduce the error rate. 4. Obtain the texture and color characteristics of the bushing at the wired terminal port and its relative position relationship with the target terminal line, and determine the bushing installation status by integrating multiple aspects of information. This can comprehensively evaluate the compliance and quality of the bushing installation, detect installation anomalies in a timely manner, and ensure the overall stability and reliability of the terminal line wiring in the panel cabinet. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flow chart of a method for identifying the wiring status of terminal wires in a panel cabinet based on semantic segmentation disclosed in an embodiment of the present application; Figure 2 This is a module diagram of a system for identifying the wiring status of terminal wires in a panel cabinet based on semantic segmentation disclosed in an embodiment of the present application; Figure 3 This is a structural diagram of an electronic device disclosed in an embodiment of the present application.

[0025] Explanation of the reference numerals: 201, acquisition module; 202, positioning module; 203, analysis module; 204, identification module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0027] In the description of the embodiments of this application, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a concrete manner.

[0028] In the description of the embodiments of the present application, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0029] This embodiment discloses a method for identifying the wiring status of terminal wires in a panel cabinet based on semantic segmentation. Figure 1 This is a flow chart of a method for identifying the wiring status of terminal wires in a panel cabinet based on semantic segmentation disclosed in an embodiment of the present application, which is applied to a panel cabinet terminal wire wiring status identification platform, such as Figure 1 As shown, the method includes the following steps: S101, using a visible light camera, an infrared camera, and a depth camera to simultaneously photograph the screen cabinet to obtain a visible light image, an infrared thermal image, and a depth image, and fusing the visible light image, the infrared thermal image, and the depth image to obtain a fused image; S102, acquiring target features from the fused image, matching the target features with preset terminal strip template features to obtain a preliminary positioning result, and processing the preliminary positioning result using geometric constraints to obtain a fused image of a terminal strip region of interest; S103, performing semantic segmentation and context analysis on the fused image of the terminal block region of interest to obtain a pixel segmentation region of the target terminal port, obtaining terminal line features from the pixel segmentation region, performing similarity matching on the terminal line features and samples in a preset database to determine a determination threshold, and determining the connection / disconnection state of the target terminal port according to the determination threshold; S104: Acquire texture features, color features, and relative positional relationship between the bushing of the connected terminal port and a target terminal wire, and determine the installation state of the bushing of the connected terminal port, where the target terminal wire is the terminal wire connected to the bushing.

[0030] A visible light camera, an infrared camera, and a depth camera are used to simultaneously capture the cabinet's surface. The visible light camera captures appearance information, including color and shape; the infrared camera captures infrared thermal images of the cabinet, reflecting the temperature distribution of different parts of the cabinet; and the depth camera captures depth information, namely the distance between each part and the camera. The captured visible light, infrared, and depth images are fused. The goal of fusion is to leverage the strengths of each image type to produce a fused image rich in information. For example, visible light images provide clear appearance details, infrared images reveal potential temperature anomalies (which may indicate wiring faults), and depth images help determine the spatial position of objects, providing a more comprehensive and accurate data foundation for subsequent terminal strip identification and analysis. Target features are extracted from the fused image. These features may include the shape, edges, and texture of the terminal strip. The extracted target features are then matched against pre-defined terminal strip template features. Template features are a pre-defined set of features representing a standard terminal strip. This matching allows for a preliminary determination of the terminal strip's position in the fused image, resulting in a preliminary positioning result. Because this preliminary positioning result may contain some error, geometric constraints are used to address this. Geometric constraints are set based on the actual layout and spatial relationships of the cabinet terminal strips, such as the relative position and spacing between terminal strips. By applying these constraints, inaccurate positioning results can be eliminated, further refining the terminal strip's location and generating a fused image of the terminal strip's region of interest. This fused image focuses on the terminal strip area, reducing the amount of data required for subsequent processing and improving efficiency. Semantic segmentation is performed on the fused image of the terminal strip's region of interest, assigning each pixel in the image to a different semantic category, such as terminal port or background, to obtain a pixel-wise segmented region of the target terminal port. Contextual analysis is also performed, considering the spatial and visual relationship between the target terminal port and its surroundings (such as adjacent terminal ports and wires) to more accurately determine the terminal port's location and boundaries. Terminal line features are extracted from the pixel-wise segmented region of the target terminal port. These features may include color, shape, thickness, and other features. The extracted terminal line features are then matched against samples in a pre-set database for similarity. This database contains a large number of sample features of terminal lines with known connection states (connected or disconnected). By calculating the similarity between the current terminal line features and the sample features, a judgment threshold is determined. Based on the judgment threshold, the connection status of the current target terminal port is determined. If the similarity between the terminal wire feature and the connection sample exceeds the judgment threshold, the connection state of the target terminal port is determined to be connected; conversely, if the similarity with the disconnection sample exceeds the judgment threshold, the connection state is determined to be disconnected. The texture features and color features of the sleeve of the connected terminal port and its relative position relationship with the target terminal wire are obtained.Texture features can reflect information such as the roughness and pattern of the bushing's surface; color features describe the bushing's color properties; and relative positional relationships clarify the specific spatial location of the bushing and the connected terminal wire. These features are compared with preset standards for normal installation conditions. If the texture features, color features, and relative positional relationships are all within the corresponding preset ranges, the bushing installation complies with specifications and the bushing's installation status at the connected terminal port is determined to be normal. Otherwise, if any feature exceeds the preset range, the bushing's installation status is determined to be abnormal. This multi-feature comprehensive judgment method enables a more comprehensive and accurate assessment of bushing installation quality, promptly identifying potential installation issues.

[0031] Optionally, fusing the visible light image, the infrared thermal image, and the depth image to obtain a fused image includes: Aligning the visible light image, the infrared thermal image, and the depth image based on a feature point registration method; Acquiring visual features of the visible light image, thermal distribution features of the infrared thermal image, and geometric features of the depth image; The visual features, the thermal distribution features, and the geometric features are spliced ​​to form a fusion feature vector, and the fusion image is obtained according to the fusion feature vector.

[0032] In image processing, feature points are points with unique properties within an image, such as corners and edges. Feature point-based registration methods first extract these representative feature points from the visible light image, infrared thermal image, and depth image. For example, in a visible light image, edge intersections and screw holes on a terminal block can serve as feature points; in an infrared thermal image, boundary points of areas with significant temperature differences can serve as feature points; and in a depth image, points where the depth of an object's surface varies dramatically can serve as feature points. By analyzing the correspondence between feature points in different images, the transformation parameters (such as translation, rotation, and scaling) between the images are calculated. These parameters are then used to transform the images so that the three images are spatially aligned. The goal of alignment is to ensure that the same target area in different images corresponds precisely, providing an accurate spatial basis for subsequent feature fusion. For example, after alignment, a terminal port on a terminal block in the visible light image, the corresponding area with abnormal temperature in the infrared thermal image, and the depth information of that terminal port in the depth image can be accurately associated. Visible light images contain rich visual information, and visual features can be extracted from multiple aspects. For example, color features—terminal blocks and wires of different colors have different pixel value distributions in visible light images. Texture features—the texture of the terminal block surface and the thickness and texture of the wires—can be extracted through algorithms. Shape features, such as the shape of the terminal opening and the routing of the wires, are also important visual features. These visual features can intuitively reflect the appearance of the cabinet terminal block and its wiring. Infrared thermal images primarily reflect the temperature distribution of an object. Thermal distribution features can include temperature peaks and valleys, as well as temperature gradients. For example, in a properly functioning cabinet, the temperatures of different components should be within a certain range. If an area experiences abnormally high or low temperatures, analyzing thermal distribution features can quickly locate the problem. Furthermore, temperature gradients can reveal the direction and speed of heat transfer, helping to identify issues such as local overheating or poor heat dissipation. Depth images provide distance information between the object and the camera, and geometric features are extracted based on this distance information. For example, the relative position, distance, and three-dimensional outline of the shape of components on the terminal block can be used to determine the spatial layout and structure of the cabinet terminal block, and to determine whether there is interference between components and whether the installation is correct. The visual features of the acquired visible light image, the thermal distribution features of the infrared image, and the geometric features of the depth image are combined to form a fused feature vector. This fused feature vector contains rich information from different image modalities, providing a more comprehensive description of the status of the cabinet terminal block and its wiring. For example, the fused feature vector includes both appearance information such as the terminal block's color and shape, as well as temperature distribution and spatial location information. A fused image is generated based on the fused feature vector.A fused image isn't simply a superposition of three original images. Instead, a specific algorithm maps the information in the fused feature vector to the image's pixel values, allowing the fused image to simultaneously present features from different modal images. For example, within a fused image, different features can be represented by varying colors, brightness, or textures, allowing the viewer to intuitively visualize the comprehensive status of the cabinet terminal block and its wiring, facilitating subsequent analysis and processing, such as terminal block identification and fault detection.

[0033] A feature-point-based registration method aligns visible light images, infrared thermal images, and depth images. This method accurately matches corresponding feature points in the different modal images, eliminating spatial deviations between the images and ensuring spatial consistency of image elements during subsequent fusion processing. This provides a precise foundation for subsequent feature fusion and avoids information misleading due to image misalignment. Visual features (such as color, texture, and edges) are captured from the visible light image; thermal distribution features (reflecting the temperature distribution of the object) are captured from the infrared thermal image; and geometric features (including information such as the spatial position, shape, and distance of the object) are captured from the depth image. By extracting features from multiple modal images, comprehensive information about the cabinet's appearance, temperature, and spatial structure is captured, providing rich and complementary feature data for subsequent fusion. The extracted visual, thermal, and geometric features are then concatenated to form a fused feature vector. This simple and efficient feature fusion method integrates feature information from different modal images into a unified vector, facilitating subsequent processing and analysis. A fused image is obtained based on the fused feature vector. The fused image integrates the feature information of multiple modal images. Compared with a single modal image, it can provide more comprehensive, accurate and rich information, which helps to more accurately identify and analyze the terminal blocks and their wiring status in the subsequent panel cabinet, thereby improving the accuracy and reliability of the recognition system.

[0034] Optionally, the feature point-based registration method for aligning the visible light image, the infrared thermal image, and the depth image includes: Acquire a first key point and a first feature descriptor of the visible light image, a second key point and a second feature descriptor of the infrared thermal image, and a third key point and a third feature descriptor of the depth image; Performing a first matching operation on the first feature descriptor and the second feature descriptor to obtain a first matching pair, performing a second matching operation on the first feature descriptor and the third feature descriptor to obtain a second matching pair, calculating a first transformation matrix from the infrared thermal image to the visible light image based on the first matching pair, and calculating a second transformation matrix from the depth image to the visible light image based on the second matching pair; The infrared thermal image is converted into the coordinate system of the visible light image by applying the first transformation matrix, the depth image is converted into the coordinate system of the visible light image by applying the second transformation matrix, and pixel values ​​of the converted images are interpolated using an interpolation method.

[0035] Extract first keypoints and first feature descriptors from the visible light image. First keypoints are typically pixels with significant features (such as corners and edges) in the visible light image. These pixels remain relatively stable despite certain image changes (such as rotation, scaling, and translation). The first feature descriptor is a quantitative representation of the features of the area surrounding the first keypoint. It uniquely describes the feature information of the first keypoint. Algorithms such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Features) can extract robust feature descriptors. Similar processing is performed on the infrared thermal image to obtain second keypoints and second feature descriptors. Infrared thermal images reflect the temperature distribution of objects. The extraction method for second keypoints and second feature descriptors is similar to that for visible light images, but focuses on features related to temperature changes. Similarly, extract third keypoints and third feature descriptors from the depth image. The depth image contains distance information between the object and the camera. The third keypoints and third feature descriptors focus on the spatial structure and distance characteristics of the object. Perform a first match between the first feature descriptor of the visible light image and the second feature descriptor of the infrared thermal image. The matching process typically uses similarity metrics between feature descriptors, such as Euclidean distance or Hamming distance. By calculating the distances between descriptors, the most similar feature point pairs are found, forming first matching pairs. Based on these first matching pairs, a first transformation matrix is ​​calculated from the infrared thermal image to the visible light image. This transformation matrix describes the spatial transformation between the infrared thermal image and the visible light image, including operations such as translation, rotation, and scaling. This matrix can be used to map points in the infrared thermal image to the coordinate system of the visible light image. A second matching is performed between the first feature descriptor of the visible light image and the third feature descriptor of the depth image, resulting in a second matching pair. Based on this second matching pair, a second transformation matrix is ​​calculated from the depth image to the visible light image. This transformation matrix is ​​used to transform points in the depth image to the coordinate system of the visible light image, ensuring spatial alignment between the depth image and the visible light image. The first transformation matrix is ​​applied to transform the infrared thermal image to the coordinate system of the visible light image, while the second transformation matrix is ​​applied to transform the depth image to the coordinate system of the visible light image. After the transformation, the pixel positions of the infrared thermal image and depth image correspond to those of the visible light image. However, due to the potential for non-integer pixel position mapping during the transformation process, some pixel positions in the transformed image do not have corresponding original pixel values. To obtain a fully aligned image, interpolation methods are used to reconstruct the pixel values ​​of the transformed image. Common interpolation methods include nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation.Nearest neighbor interpolation directly selects the original pixel value closest to the target pixel as the interpolation result. This method is computationally simple but has low accuracy. Bilinear interpolation calculates the interpolation result by taking a weighted average of the four neighboring pixels surrounding the target pixel, resulting in a smoother image. Bicubic interpolation considers more neighboring pixels, resulting in better interpolation results but requiring a relatively high computational load. Through interpolation, a fused image aligned in the visible light image coordinate system is obtained. This image combines visible light, infrared, and depth information, providing a more accurate data foundation for subsequent image analysis and processing.

[0036] Keypoints and feature descriptors are obtained for the visible light image, infrared thermal image, and depth image, respectively. Keypoints are points in an image with significant characteristics and stability, such as corners and edges. Feature descriptors are vectors that encode the features of the local area surrounding a keypoint, accurately describing the keypoint's characteristic information. This provides a precise feature basis for subsequent image matching and registration. The feature descriptors of the visible light image are matched with those of the infrared thermal image and depth image, respectively, to obtain a first matching pair and a second matching pair. This feature descriptor-based matching method offers high accuracy and robustness, enabling accurate correspondence between images of different modalities, even with differences in brightness, contrast, noise, and other factors. The first transformation matrix from the infrared thermal image to the visible light image is calculated based on the first matching pair, and the second transformation matrix from the depth image to the visible light image is calculated based on the second matching pair. The transformation matrix describes the spatial transformation relationship between images. By calculating the transformation matrix, the coordinate correspondence between images of different modalities can be accurately determined, providing a mathematical basis for subsequent image alignment. Due to the high accuracy of feature matching, the transformation matrix calculated based on the matching pairs also has high precision, effectively eliminating spatial deviations between the different modal images and achieving precise image alignment. A first transformation matrix is ​​used to transform the infrared thermal image into the visible light image coordinate system, while a second transformation matrix is ​​used to transform the depth image into the visible light image coordinate system. This coordinate system transformation enables spatial alignment of the different modal images, providing a unified coordinate basis for subsequent image fusion and analysis. Interpolation methods are used to interpolate pixel values ​​in the transformed images. Since pixel positions in the images may not be integer coordinates after the image coordinate system transformation, interpolation methods are required to estimate the pixel values ​​at these locations. The selection and implementation of the interpolation method ensures the quality of the interpolated image, making the aligned image visually smoother and more natural, while avoiding information loss or distortion caused by pixel position mismatches. After this registration and alignment process, the different modal images are precisely aligned in terms of spatial position and feature information, providing a good foundation for subsequent multimodal image fusion. The fused image leverages the strengths of different modal images to provide more comprehensive, accurate, and rich information, helping to improve the recognition and analysis of cabinet terminal blocks and their wiring status. Precise image registration and alignment can reduce errors and uncertainties in subsequent image processing and analysis, improving the overall performance and stability of the system. For example, in tasks such as object detection, feature extraction, and classification, registered and aligned images can provide more accurate data, thereby improving the accuracy and reliability of detection and classification.

[0037] Optionally, performing semantic segmentation and context analysis on the fused image of the terminal block region of interest to obtain the pixel segmentation region of the target terminal port includes: Using a semantic segmentation algorithm to classify the target pixel in the fused image of the terminal strip region of interest, and determine whether the target pixel belongs to the target terminal opening region, so as to obtain a preliminary semantic segmentation result, wherein the target pixel is any pixel in the fused image of the terminal strip region of interest; Analyzing the distance and relative position relationship between a target terminal port and adjacent terminal ports, as well as the brightness and contrast of the region where the target terminal port is located, to adjust the preliminary semantic segmentation result, wherein the target terminal port is any terminal port in the preliminary semantic segmentation result; The pixel segmentation area of ​​the target terminal port is constructed according to the adjusted preliminary semantic segmentation result.

[0038] A semantic segmentation algorithm is used to classify each target pixel in the fused image of the terminal strip's region of interest. Semantic segmentation is an image processing technique that assigns image pixels to different semantic categories based on pixel features (such as color, texture, and shape). In this scenario, the goal is to classify pixels as belonging to or not belonging to the target terminal port area. By performing classification judgment on each target pixel, a preliminary semantic segmentation result is obtained. This result is a binary image (or similar representation), in which pixels belonging to the target terminal port area are labeled with one category (for example, white) and pixels not belonging to the target terminal port area are labeled with another category (for example, black). This step preliminarily identifies the region in the image that may belong to the target terminal port. However, due to the limitations of the algorithm itself and potential noise and interference in the image, the preliminary results may contain some errors. Contextual information such as the distance and relative position between the target terminal port and adjacent terminals, as well as the brightness and contrast of the target terminal port area, is analyzed. Terminal ports in a terminal strip typically have a certain layout pattern, and the distance and relative position between adjacent terminals are important contextual information. For example, if the distance between two terminal openings is too small, and the initial semantic segmentation results separate them, a segmentation error may have occurred. Conversely, if the distance is too small but they are segmented into a single region, this may also be an error. By analyzing this information, the terminal opening boundaries in the initial semantic segmentation results can be adjusted to better align with the actual terminal strip layout. The brightness and contrast of the target terminal opening region can also affect the semantic segmentation results. For example, if the brightness and contrast of the target terminal opening region differ significantly from the surrounding area, but the initial semantic segmentation results do not accurately capture this difference, resulting in inaccurate segmentation, analyzing the brightness and contrast information can correct the segmentation boundaries to make the segmentation results more consistent with actual visual characteristics. Based on the above contextual analysis, the initial semantic segmentation results are adjusted. Adjustments may include expanding or reducing the target terminal opening region or correcting the terminal opening boundaries to improve semantic segmentation accuracy. Based on the adjusted initial semantic segmentation results, a pixel segmentation region for the target terminal opening is constructed. This step converts the adjusted binary image (or similar representation) into a specific set of pixels, specifically identifying which pixels belong to the target terminal opening region. The constructed pixel segmentation area can be used for subsequent tasks such as terminal port feature extraction and wiring status judgment, providing basic data for accurate identification and analysis of panel cabinet terminal blocks and their wiring status.

[0039] A semantic segmentation algorithm is used to classify target pixels in the fused image of the terminal strip's region of interest, determining whether they belong to the target terminal port area, thereby obtaining preliminary semantic segmentation results. This process meticulously assigns each pixel in the image to a corresponding category (terminal port or non-terminal port), achieving preliminary localization and demarcation of the terminal port area, providing a foundation for subsequent analysis. Compared to traditional methods based on simple thresholding or edge detection, semantic segmentation algorithms are more capable of handling the varying shapes and sizes of terminal ports in images, as well as complex backgrounds, improving pixel classification accuracy. During the semantic segmentation process, the algorithm comprehensively considers multiple features, such as color, texture, and shape, within the image pixel, leveraging the image's visual information to classify pixels. This multi-feature fusion classification approach provides a more comprehensive description of the terminal port's characteristics, reducing misclassifications caused by the limitations of a single feature and improving the reliability of the preliminary semantic segmentation results. The preliminary semantic segmentation results are adjusted by analyzing contextual information, such as the distance and relative position between the target terminal port and adjacent terminals, as well as the brightness and contrast of the target terminal port's area. For example, by analyzing the positional relationship between adjacent terminal openings, it is possible to determine the rationality of the terminal opening boundary in the initial segmentation results, avoiding inaccurate boundary delineation caused by factors such as local noise or image blur. Furthermore, considering brightness and contrast information can better distinguish the terminal opening from the background. This can further optimize the segmentation boundary, making the segmentation results more realistic, especially when the terminal opening and background are of similar color or when the lighting is uneven. The introduction of contextual analysis enhances the segmentation algorithm's adaptability to complex scenes. In practical applications, cabinet terminal strip images may be affected by a variety of factors, such as lighting variations, occlusions, and noise. By incorporating contextual information, the algorithm can better cope with these interfering factors, reduce the sensitivity of the segmentation results to image quality variations, and improve the robustness and stability of the segmentation. Based on the adjusted initial semantic segmentation results, the pixel segmentation region of the target terminal opening is constructed, accurately determining the terminal opening's location and range in the image. This is crucial for subsequent terminal opening analysis, such as terminal wire feature extraction and wiring status determination. Accurate pixel segmentation regions ensure that subsequent processing is performed only on the terminal opening region, reducing interference from irrelevant information and improving processing efficiency and accuracy. The constructed pixel segmentation region of the target terminal port provides foundational data for subsequent image analysis and processing tasks. For example, when determining the on / off status of a terminal port, the terminal wire features can be accurately extracted from the pixel segmentation region, avoiding the blind search and feature extraction from the entire image. This improves the accuracy and efficiency of feature extraction, thereby enhancing the performance of the entire wiring status recognition system.

[0040] Optionally, performing similarity matching between the terminal wire feature and samples in a preset database to determine a determination threshold, and determining the connection status of the target terminal port according to the determination threshold includes: Comparing and calculating the terminal wire features with each sample feature in a preset database one by one to obtain a similarity value set; Dividing the similarity values ​​in the similarity value set into a first similarity value set and a second similarity value set according to the connection on / off state labels; determining a determination threshold according to a first distribution characteristic of similarity values ​​in the first similarity value set and a second distribution characteristic of similarity values ​​in the second similarity value set; When the average similarity value in the target similarity value set in the similarity set is greater than or equal to the judgment threshold, the wiring of the target terminal port is determined to be in an on state; otherwise, the wiring of the target terminal port is determined to be in a disconnected state. The target similarity value set is a set of similarity values ​​whose similarity values ​​are higher than a preset threshold.

[0041] The extracted target terminal wire features are compared one by one with each sample feature in a preset database, and similarity is calculated. The preset database contains a large number of sample features of terminal wires with known connection states (connected or disconnected). Using specific similarity calculation methods (such as Euclidean distance or cosine similarity), a set of similarity values ​​is generated. Each value in this set represents the degree of similarity between the target terminal wire features and a sample feature in the database. This process ensures that all possible samples are fully considered, providing a rich data foundation for subsequent determination of the judgment threshold. The similarity values ​​in the set are divided into a first similarity set and a second similarity set based on the connection and disconnection state labels. For example, the similarity values ​​calculated between sample features in the connected state and the target terminal wire features can be grouped into the first similarity set, while the similarity values ​​calculated between sample features in the disconnected state and the target terminal wire features can be grouped into the second similarity set. This division allows for separate analysis of the distribution of similarity values ​​in the connected and disconnected states, providing a basis for determining an appropriate judgment threshold. A decision threshold is determined based on first distribution characteristics (such as mean, variance, and distribution range) of the similarity values ​​in the first similarity value set and second distribution characteristics of the similarity values ​​in the second similarity value set. Typically, the distribution of similarity values ​​in the on state differs from that in the off state. For example, similarity values ​​in the on state may be generally higher and more concentrated, while similarity values ​​in the off state may be generally lower and more dispersed. By analyzing these distribution characteristics, an appropriate threshold can be found that effectively distinguishes the similarity values ​​between the on and off states. For example, a threshold can be selected that lies outside the overlapping region of the two distributions to minimize the possibility of false positives. Similarity values ​​with similarity values ​​above a preset threshold are filtered from the similarity value set to form a target similarity value set. The average similarity value of the target similarity value set is calculated and compared with the previously determined decision threshold. If the average similarity value is greater than or equal to the decision threshold, the target terminal port is determined to be connected; otherwise, the target terminal port is determined to be disconnected. By comparing the average similarity value of the target similarity value set with the judgment threshold, the on-off status of the target terminal port is automatically judged.

[0042] By comparing and calculating each terminal wire feature against each sample feature in a pre-set database, a set of similarity values ​​is generated, accurately quantifying the degree of similarity between the terminal wire features and the sample features. This quantification provides an objective and measurable basis for subsequent wiring status judgment, avoiding the uncertainty that can arise from relying solely on subjective experience. The sample features in the pre-set database cover terminal wire features under different wiring conditions. By comparing these samples, a large amount of similarity data is accumulated. This data is not only used to determine the current wiring status but also provides rich material for subsequent algorithm optimization and model training, helping to improve the overall performance and adaptability of the system. The similarity value set is divided into two subsets based on the wiring connection and disconnection status labels, and the judgment threshold is determined based on the distribution characteristics (such as mean, variance, and distribution range) of the similarity values ​​in the two subsets. This approach fully considers the actual distribution of feature similarity under different wiring conditions, making the threshold setting more scientific and reasonable, effectively distinguishing between the connected and disconnected states, and reducing the false positive rate. Because the threshold is determined based on the distribution characteristics of actual sample data, when the sample characteristics in the database change (such as the addition of samples of different terminal wire types), the threshold can be adjusted accordingly to adapt to the new data distribution, ensuring the accuracy and stability of the system in different scenarios. Using the determined judgment threshold as the standard, the connection status of the target terminal port is accurately determined by comparing the average similarity value of the target similarity value set (the set with similarity values ​​above the preset threshold) with the judgment threshold. This judgment method, based on objective data comparison, avoids human interference in the judgment results, and improves the accuracy and reliability of wiring status judgment.

[0043] Optionally, acquiring the texture features, color features, and relative positional relationship between the sleeve of the connected terminal port and the terminal wire to determine the installation state of the sleeve of the connected terminal port includes: The target area image of the casing is processed using a local binary pattern algorithm to calculate the LBP value of each pixel, and the texture feature vector of the casing is determined according to the total LBP value of the target area image; Converting the target area image from the RGB color space to the HSV color space, respectively calculating the color histograms of the target area image in three channels, namely, hue, saturation, and lightness, and concatenating the color histograms of the three channels to form a color feature vector of the sleeve; Calculating the Euclidean distance between the center point of the sleeve and the center point of the target terminal line, and the vertical distance from the center point of the sleeve to the straight line on which the target terminal line is located, and determining the relative positional relationship between the sleeve and the target terminal line according to the Euclidean distance and the vertical distance; If the texture feature vector, the color feature vector, and the relative position relationship are all within corresponding preset ranges, it is determined that the installation state of the sleeve is normal; otherwise, it is determined that the installation state of the sleeve is abnormal.

[0044] The target area image of the casing is processed using the Local Binary Pattern (LBP) algorithm. The LBP algorithm is a simple yet effective texture description operator that extracts texture information by comparing the grayscale values ​​of a central pixel with those of its surrounding neighborhood pixels. For each pixel in the target area image, a neighborhood (e.g., a 3×3 neighborhood) is selected with the central pixel as the center. The grayscale value of the central pixel is used as a threshold and compared with the grayscale values ​​of the eight pixels within the neighborhood. If the grayscale value of a neighboring pixel is greater than or equal to that of the central pixel, the pixel is marked as 1; otherwise, it is marked as 0. This results in an 8-bit binary number for each neighboring pixel, which, when converted to decimal, is the LBP value for that pixel. The LBP values ​​of all pixels in the target area image are calculated and statistically analyzed to obtain a feature vector that reflects the casing texture characteristics, namely the casing texture feature vector. This feature vector can be used to describe information such as the texture pattern and roughness of the casing surface. Convert the target area image from RGB color space to HSV color space. The RGB color space is based on an additive color mixing model based on the three primary colors of red, green, and blue. The HSV color space is more consistent with human color perception and consists of three components: hue, saturation, and value. Calculate the color histograms of the target area image for each of the three channels: hue, saturation, and value. A color histogram is a statistical method for analyzing the color distribution in an image. It divides the color range into several intervals and counts the number of pixels within each interval to determine the color distribution within the image. Concatenate the color histograms of the three channels to form a one-dimensional feature vector, the color feature vector of the casing. This feature vector can be used to describe information such as the casing's color distribution and color vividness. Calculate the coordinates of the center point of the casing and the center point of the target terminal wire. The center point coordinates can be obtained by performing mathematical operations on the pixels in the image. For example, for the casing area, the average of all pixel coordinates can be calculated as the center point coordinate. Calculate the Euclidean distance between the center point of the sleeve and the center point of the target terminal line, as well as the perpendicular distance from the center point of the sleeve to the line containing the target terminal line. The Euclidean distance is the straight-line distance between two points, and the perpendicular distance is the shortest distance from a point to a straight line. These two distances can be used to describe the relative spatial positional relationship between the sleeve and the target terminal line. Based on the calculated Euclidean distance and perpendicular distance, the relative positional relationship between the sleeve and the target terminal line is determined. For example, a certain distance threshold can be set. If both the Euclidean distance and the perpendicular distance are within the threshold range, the relative positional relationship between the sleeve and the target terminal line is considered normal; otherwise, the relative positional relationship is considered abnormal. Set corresponding preset ranges for the texture feature vector, color feature vector, and relative positional relationship.These preset ranges are determined based on actual needs and experience and are used to determine whether the casing is properly installed. If the casing's texture feature vector, color feature vector, and relative position are all within the corresponding preset ranges, the casing is considered properly installed; otherwise, it is considered abnormal. This approach enables automated detection and judgment of the casing's installation status, improving detection efficiency and accuracy.

[0045] The LBP algorithm effectively extracts local texture information from images and is robust to lighting variations and noise. It accurately describes the texture characteristics of the casing surface, such as texture roughness and directionality, providing important information for subsequent installation status assessment. Quantifying texture features into feature vectors makes texture feature assessment more objective and quantifiable. By comparing with a preset range, it can accurately determine whether the casing texture conforms to the texture characteristics of a normal installation state, avoiding the errors and inconsistencies that may be caused by subjective visual judgment alone. The HSV color space better aligns with human color perception and more accurately describes color properties. By integrating multi-channel color information, it can comprehensively reflect the color characteristics of the casing, including color type, vividness, and brightness. The extraction and analysis of color feature vectors helps determine whether the casing color meets the color requirements of a normal installation state. Under normal installation conditions, the casing color should be relatively stable and consistent. By comparing with the preset color feature range, abnormal casing color (such as fading or discoloration) can be promptly detected, thereby determining whether the casing is properly installed. The Euclidean distance between the center point of the bushing and the center point of the target terminal line, as well as the perpendicular distance from the bushing center point to the line containing the target terminal line, are calculated. The relative positional relationship between the bushing and the target terminal line is determined based on these two distances. The calculation of the Euclidean and perpendicular distances accurately quantifies the positional deviation between the bushing and the terminal line, providing quantitative indicators for determining whether the bushing is correctly installed. By comparing the relative positional relationship with a preset range, the bushing's installation position can be accurately assessed. If the relative positional relationship between the bushing and the terminal line exceeds the preset range, it may indicate inaccurate bushing installation, such as looseness or displacement, which may affect the terminal connection quality and electrical performance. The bushing's installation status is comprehensively assessed by integrating three aspects: texture feature vectors, color feature vectors, and relative positional relationship information. This multi-dimensional, comprehensive assessment method enables more accurate judgment of the bushing's installation status, avoiding the limitations of single-feature judgment. If the texture feature vector, color feature vector, and relative positional relationship are all within the corresponding preset ranges, the bushing's installation status is determined to be normal; otherwise, it is determined to be abnormal. This judgment method is simple and efficient, and can quickly and accurately detect abnormal bushing installation status, providing timely and accurate basis for equipment maintenance and repair, helping to ensure the normal operation of the equipment and reduce electrical failures and safety hazards caused by abnormal bushing installation.

[0046] Optionally, the method further includes: A knowledge graph of the wiring status of the terminal wires in the panel cabinet is constructed, using the terminal port, terminal wire, and bushing as entity nodes, and the wiring relationship and bushing installation relationship as the relationship edges between the entities. For terminal ports whose wiring on / off status does not conform to the wiring specifications in the knowledge graph, mark them and extract correction suggestions from the knowledge graph; For the terminal ports that have been wired, a matching analysis is performed based on the installation status of the bushing and the fault mode library in the knowledge graph to predict potential failure risks, and corresponding early warning information is given based on the potential failure risks.

[0047] Terminal ports, terminal wires, and bushings are used as entity nodes in the knowledge graph. Terminal ports are key electrical connections, used to connect or remove wires; terminal wires are wires that connect terminal ports to external circuits; and bushings typically protect terminal wires or terminal ports, providing insulation and protection. Using these as entity nodes clearly represents the main objects involved in the cabinet terminal wiring system. Wiring relationships and bushing installation relationships are used as relationship edges between entities. The wiring relationship describes the connection method between the terminal wire and the terminal port, such as direct connection or indirect connection through other components. The bushing installation relationship reflects the installation coordination between the bushing, the terminal wire, and the terminal port, such as whether the bushing is correctly fitted onto the terminal wire and whether it is tightly fitted to the terminal port. These relationship edges connect the entity nodes to form a complete knowledge graph, visually demonstrating the relationships and interactions between the various components in the cabinet terminal wiring system. Constructing such a knowledge graph helps structure and visualize complex cabinet terminal wiring status information, facilitating subsequent analysis and processing. It integrates various entities and relationships within the wiring system, providing a foundational knowledge framework for determining wiring compliance and predicting potential faults. Terminal ports whose wiring status does not conform to the wiring specifications in the knowledge graph are marked. Wiring specifications in the knowledge graph are based on industry standards, design requirements, and actual operational experience. When the wiring status of a terminal port is inconsistent with these specifications, it indicates a potential wiring problem that requires attention and resolution. This marking allows for quick location of these abnormal terminal ports, facilitating subsequent troubleshooting and repair. Corrective action suggestions for these abnormal terminal ports are extracted from the knowledge graph. The knowledge graph not only contains wiring specifications but also may store corrective action methods and experience for different abnormalities. When a terminal port with an abnormal wiring status is discovered, corresponding corrective action suggestions are automatically extracted based on the pre-stored information in the knowledge graph, providing maintenance personnel with specific operational guidance and improving repair efficiency and accuracy. For already wired terminal ports, the bushing installation status is matched against the fault mode library in the knowledge graph for analysis. The fault mode library is a key component of the knowledge graph. It records various possible failure modes and the corresponding bushing installation status characteristics. By matching the bushing installation status of a wired terminal port with the characteristics in the fault mode library, it is possible to determine whether the terminal port has a potential failure risk. Based on the matching analysis results of potential failure risks, corresponding early warning information is provided. If the bushing installation status of a terminal port is found to match a failure mode characteristic in the fault mode library, it indicates that the terminal port has a high potential failure risk. At this time, the system can provide timely early warning information, reminding relevant personnel to take measures for inspection and repair to prevent the occurrence or expansion of the fault and ensure the safe and stable operation of the panel cabinet terminal wiring system.

[0048] In some embodiments, based on the terminal port position of the preset template, the terminal line pixel area contained in this position is calculated to be greater than 20% of the preset terminal port position. If it is greater, the terminal port is in a connected state; otherwise, it is in an unconnected state. The minimum circumscribed rectangular frame of the connected pixel segmentation area is expanded outward by 1.25 times the width and height, and the image is cut out to perform a second-order sleeve segmentation detection. If the sleeve segmentation area / detection area is greater than 0.5, the sleeve is installed normally. Finally, the terminal port position of each preset template is output, and the connection status of each terminal port is marked. The normal / missing status of the sleeve is marked for the connected terminal port.

[0049] This embodiment also discloses a system for identifying the wiring status of terminal wires in a panel cabinet based on semantic segmentation. Figure 2 This is a module diagram of a system for identifying the wiring status of terminal wires in a panel cabinet based on semantic segmentation disclosed in an embodiment of the present application. Figure 2 As shown, the system includes a collection module 201, a positioning module 202, an analysis module 203 and an identification module 204, wherein: The acquisition module 201 is configured to use a visible light camera, an infrared camera, and a depth camera to simultaneously shoot the screen cabinet, obtain a visible light image, an infrared thermal image, and a depth image, and fuse the visible light image, the infrared thermal image, and the depth image to obtain a fused image; A positioning module 202 is configured to obtain target features from the fused image, match the target features with preset terminal strip template features to obtain a preliminary positioning result, and process the preliminary positioning result using geometric constraints to obtain a fused image of the terminal strip region of interest; An analysis module 203 is configured to perform semantic segmentation and context analysis on the fused image of the terminal block region of interest to obtain a pixel segmentation region of a target terminal port, obtain terminal line features from the pixel segmentation region, perform similarity matching on the terminal line features with samples in a preset database to determine a determination threshold, and determine the connection / disconnection state of the target terminal port based on the determination threshold; The identification module 204 is configured to obtain the texture characteristics, color characteristics and relative position relationship of the sleeve of the connected terminal port and the target terminal wire, and determine the installation status of the sleeve of the connected terminal port, and the target terminal wire is the terminal wire connected to the sleeve.

[0050] Optionally, the acquisition module 201 is configured to: Aligning the visible light image, the infrared thermal image, and the depth image based on a feature point registration method; Acquiring visual features of the visible light image, thermal distribution features of the infrared thermal image, and geometric features of the depth image; The visual features, the thermal distribution features, and the geometric features are spliced ​​to form a fusion feature vector, and the fusion image is obtained according to the fusion feature vector.

[0051] Optionally, the acquisition module 201 is configured to: Acquire a first key point and a first feature descriptor of the visible light image, a second key point and a second feature descriptor of the infrared thermal image, and a third key point and a third feature descriptor of the depth image; Performing a first matching operation on the first feature descriptor and the second feature descriptor to obtain a first matching pair, performing a second matching operation on the first feature descriptor and the third feature descriptor to obtain a second matching pair, calculating a first transformation matrix from the infrared thermal image to the visible light image based on the first matching pair, and calculating a second transformation matrix from the depth image to the visible light image based on the second matching pair; The infrared thermal image is converted into the coordinate system of the visible light image by applying the first transformation matrix, the depth image is converted into the coordinate system of the visible light image by applying the second transformation matrix, and pixel values ​​of the converted images are interpolated using an interpolation method.

[0052] Optionally, the analysis module 203 is configured to: Using a semantic segmentation algorithm to classify the target pixel in the fused image of the terminal strip region of interest, and determine whether the target pixel belongs to the target terminal opening region, so as to obtain a preliminary semantic segmentation result, wherein the target pixel is any pixel in the fused image of the terminal strip region of interest; Analyzing the distance and relative position relationship between a target terminal port and adjacent terminal ports, as well as the brightness and contrast of the region where the target terminal port is located, to adjust the preliminary semantic segmentation result, wherein the target terminal port is any terminal port in the preliminary semantic segmentation result; The pixel segmentation area of ​​the target terminal port is constructed according to the adjusted preliminary semantic segmentation result.

[0053] Optionally, the analysis module 203 is configured to: Comparing and calculating the terminal wire features with each sample feature in a preset database one by one to obtain a similarity value set; Dividing the similarity values ​​in the similarity value set into a first similarity value set and a second similarity value set according to the connection on / off state labels; determining a determination threshold according to a first distribution characteristic of similarity values ​​in the first similarity value set and a second distribution characteristic of similarity values ​​in the second similarity value set; When the average similarity value in the target similarity value set in the similarity set is greater than or equal to the judgment threshold, the wiring of the target terminal port is determined to be in an on state; otherwise, the wiring of the target terminal port is determined to be in a disconnected state. The target similarity value set is a set of similarity values ​​whose similarity values ​​are higher than a preset threshold.

[0054] Optionally, the identification module 204 is configured to: The target area image of the casing is processed using a local binary pattern algorithm to calculate the LBP value of each pixel, and the texture feature vector of the casing is determined according to the total LBP value of the target area image; Converting the target area image from the RGB color space to the HSV color space, respectively calculating the color histograms of the target area image in three channels, namely, hue, saturation, and lightness, and concatenating the color histograms of the three channels to form a color feature vector of the sleeve; Calculating the Euclidean distance between the center point of the sleeve and the center point of the target terminal line, and the vertical distance from the center point of the sleeve to the straight line on which the target terminal line is located, and determining the relative positional relationship between the sleeve and the target terminal line according to the Euclidean distance and the vertical distance; If the texture feature vector, the color feature vector, and the relative position relationship are all within corresponding preset ranges, it is determined that the installation state of the sleeve is normal; otherwise, it is determined that the installation state of the sleeve is abnormal.

[0055] Optionally, the system further includes a map module, wherein the map module is configured to: A knowledge graph of the wiring status of the terminal wires in the panel cabinet is constructed, using the terminal port, terminal wire, and bushing as entity nodes, and the wiring relationship and bushing installation relationship as the relationship edges between the entities. For terminal ports whose wiring on / off status does not conform to the wiring specifications in the knowledge graph, mark them and extract correction suggestions from the knowledge graph; For the terminal ports that have been wired, a matching analysis is performed based on the installation status of the bushing and the fault mode library in the knowledge graph to predict potential failure risks, and corresponding early warning information is given based on the potential failure risks.

[0056] It should be noted that the above embodiments provide devices that implement their functions using only the division of the above functional modules as examples. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0057] This embodiment also discloses an electronic device, referring to Figure 3 The electronic device may include: at least one processor 301 , at least one communication bus 302 , a user interface 303 , a network interface 304 , and at least one memory 305 .

[0058] The communication bus 302 is used to implement the connection and communication between these components.

[0059] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0060] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0061] The processor 301 may include one or more processing cores. The processor 301 utilizes various interfaces and circuits to connect various components within the server. It executes instructions, programs, code sets, or instruction sets stored in the memory 305, as well as accesses data stored in the memory 305, to perform various server functions and process data. Optionally, the processor 301 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 301 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing content displayed on the display screen; and the modem handles wireless communications. It is understood that the modem may also be implemented as a separate chip, rather than integrated into the processor 301.

[0062] Among them, the memory 305 may include a random access memory 305 (Random Access Memory, RAM), and may also include a read-only memory 305 (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may also be optionally at least one storage device located away from the aforementioned processor 301. As Figure 3 As shown, the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface 303 module, and an application program for a method for identifying the wiring status of terminal wires of a panel cabinet based on semantic segmentation.

[0063] exist Figure 3 In the electronic device shown, the user interface 303 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 301 can be used to call an application stored in the memory 305 that contains a method for identifying the wiring status of the terminal wires of the screen cabinet based on semantic segmentation. When executed by one or more processors 301, the electronic device executes one or more methods in the above embodiments.

[0064] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0065] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0066] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any variations, uses or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the technical field that are not recorded in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for identifying the wiring status of panel cabinet terminals based on semantic segmentation, characterized in that: Applied to a panel cabinet terminal line connection status identification platform, the method includes: Use a visible light camera, an infrared camera, and a depth camera to simultaneously shoot the screen cabinet to obtain a visible light image, an infrared thermal image, and a depth image, and fuse the visible light image, the infrared thermal image, and the depth image to obtain a fused image; Acquiring target features from the fused image, matching the target features with preset terminal strip template features to obtain a preliminary positioning result, and processing the preliminary positioning result using geometric constraints to obtain a fused image of a terminal strip region of interest; Performing semantic segmentation and context analysis on the fused image of the terminal block region of interest to obtain a pixel segmentation region of the target terminal port, acquiring terminal line features from the pixel segmentation region, performing similarity matching on the terminal line features with samples in a preset database to determine a judgment threshold, and determining the connection and disconnection status of the target terminal port based on the judgment threshold; The texture features, color features, and relative positional relationship between the sleeve of the connected terminal port and the target terminal wire are obtained to determine the installation state of the sleeve of the connected terminal port, where the target terminal wire is the terminal wire connected to the sleeve.

2. The method for identifying the connection status of terminal wires of a panel cabinet based on semantic segmentation according to claim 1 is characterized in that: The fusing the visible light image, the infrared thermal image, and the depth image to obtain a fused image includes: Aligning the visible light image, the infrared thermal image, and the depth image based on a feature point registration method; Acquiring visual features of the visible light image, thermal distribution features of the infrared thermal image, and geometric features of the depth image; The visual features, the thermal distribution features, and the geometric features are spliced ​​to form a fusion feature vector, and the fusion image is obtained according to the fusion feature vector.

3. The method for identifying the connection status of terminal wires of a panel cabinet based on semantic segmentation according to claim 2 is characterized in that: The feature point-based registration method, aligning the visible light image, the infrared thermal image, and the depth image, includes: Acquire a first key point and a first feature descriptor of the visible light image, a second key point and a second feature descriptor of the infrared thermal image, and a third key point and a third feature descriptor of the depth image; Performing a first matching operation on the first feature descriptor and the second feature descriptor to obtain a first matching pair, performing a second matching operation on the first feature descriptor and the third feature descriptor to obtain a second matching pair, calculating a first transformation matrix from the infrared thermal image to the visible light image based on the first matching pair, and calculating a second transformation matrix from the depth image to the visible light image based on the second matching pair; The infrared thermal image is converted into the coordinate system of the visible light image by applying the first transformation matrix, the depth image is converted into the coordinate system of the visible light image by applying the second transformation matrix, and pixel values ​​of the converted images are interpolated using an interpolation method.

4. The method for identifying the connection status of terminal wires of a panel cabinet based on semantic segmentation according to claim 1 is characterized in that: The semantic segmentation and context analysis of the fused image of the terminal block region of interest to obtain the pixel segmentation region of the target terminal port includes: Using a semantic segmentation algorithm to classify the target pixel in the fused image of the terminal strip region of interest, and determine whether the target pixel belongs to the target terminal opening region, so as to obtain a preliminary semantic segmentation result, wherein the target pixel is any pixel in the fused image of the terminal strip region of interest; Analyzing the distance and relative position relationship between a target terminal port and adjacent terminal ports, as well as the brightness and contrast of the region where the target terminal port is located, to adjust the preliminary semantic segmentation result, wherein the target terminal port is any terminal port in the preliminary semantic segmentation result; The pixel segmentation area of ​​the target terminal port is constructed according to the adjusted preliminary semantic segmentation result.

5. The method for identifying the connection status of terminal wires of a panel cabinet based on semantic segmentation according to claim 4 is characterized in that: The similarity matching of the terminal wire feature with samples in a preset database to determine a determination threshold, and determining the connection on / off state of the target terminal port according to the determination threshold includes: Comparing and calculating the terminal wire features with each sample feature in a preset database one by one to obtain a similarity value set; Dividing the similarity values ​​in the similarity value set into a first similarity value set and a second similarity value set according to the connection on / off state labels; determining a determination threshold according to a first distribution characteristic of similarity values ​​in the first similarity value set and a second distribution characteristic of similarity values ​​in the second similarity value set; When the average similarity value in the target similarity value set in the similarity set is greater than or equal to the judgment threshold, the wiring of the target terminal port is determined to be in an on state; otherwise, the wiring of the target terminal port is determined to be in a disconnected state. The target similarity value set is a set of similarity values ​​whose similarity values ​​are higher than a preset threshold.

6. The method for identifying the connection status of terminal wires of a panel cabinet based on semantic segmentation according to claim 1 is characterized in that: The acquiring of the texture features, color features, and relative positional relationship between the sleeve of the connected terminal port and the terminal wire, and determining the installation state of the sleeve of the connected terminal port includes: The target area image of the casing is processed using a local binary pattern algorithm to calculate the LBP value of each pixel, and the texture feature vector of the casing is determined according to the total LBP value of the target area image; Converting the target area image from the RGB color space to the HSV color space, respectively calculating the color histograms of the target area image in three channels, namely, hue, saturation, and lightness, and concatenating the color histograms of the three channels to form a color feature vector of the sleeve; Calculating the Euclidean distance between the center point of the sleeve and the center point of the target terminal line, and the vertical distance from the center point of the sleeve to the straight line on which the target terminal line is located, and determining the relative positional relationship between the sleeve and the target terminal line according to the Euclidean distance and the vertical distance; If the texture feature vector, the color feature vector, and the relative position relationship are all within corresponding preset ranges, it is determined that the installation state of the sleeve is normal; otherwise, it is determined that the installation state of the sleeve is abnormal.

7. The method for identifying the connection status of terminal wires of a panel cabinet based on semantic segmentation according to claim 1 is characterized in that: The method further comprises: A knowledge graph of the wiring status of the terminal wires in the panel cabinet is constructed, using the terminal port, terminal wire, and bushing as entity nodes, and the wiring relationship and bushing installation relationship as the relationship edges between the entities. For terminal ports whose wiring on / off status does not conform to the wiring specifications in the knowledge graph, mark them and extract correction suggestions from the knowledge graph; For the terminal ports that have been wired, a matching analysis is performed based on the installation status of the bushing and the fault mode library in the knowledge graph to predict potential failure risks, and corresponding early warning information is given based on the potential failure risks.

8. A system for identifying the wiring status of panel cabinet terminals based on semantic segmentation, characterized in that: It includes acquisition module, positioning module, analysis module and recognition module, among which: an acquisition module configured to simultaneously photograph the screen cabinet using a visible light camera, an infrared camera, and a depth camera to obtain a visible light image, an infrared thermal image, and a depth image, and fuse the visible light image, the infrared thermal image, and the depth image to obtain a fused image; a positioning module configured to obtain target features from the fused image, match the target features with preset terminal strip template features to obtain a preliminary positioning result, and process the preliminary positioning result using geometric constraints to obtain a fused image of the terminal strip region of interest; an analysis module configured to perform semantic segmentation and context analysis on the fused image of the terminal block region of interest to obtain a pixel segmentation region of a target terminal port, obtain terminal line features from the pixel segmentation region, perform similarity matching on the terminal line features with samples in a preset database to determine a determination threshold, and determine the connection / disconnection state of the target terminal port according to the determination threshold; The identification module is configured to obtain the texture characteristics, color characteristics and relative position relationship of the sleeve of the connected terminal port and the target terminal wire, and determine the installation status of the sleeve of the connected terminal port, and the target terminal wire is the terminal wire connected to the sleeve.

9. An electronic device, characterized in that: The electronic device comprises a processor, a memory, a user interface and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Images