Joint junction box junction state identification and diagnosis method and device

Abstract

The invention provides a method and device for identifying and diagnosing the wiring state of a combined junction box, and the method comprises the steps: collecting a multi-view two-dimensional image and multispectral three-dimensional point cloud data, and carrying out the calibration and fusion of a sensor, thereby forming a multispectral fusion point cloud; image features are decomposed into domain invariant semantic features and domain related style features by using a cross-domain feature decoupling mechanism, and a semantic segmentation result and a two-dimensional topological semantic graph are generated in combination with a double attention mechanism; constructing a three-dimensional geometric spectrum field based on implicit neural representation, and obtaining accurate three-dimensional space coordinates of a wiring assembly and a joint through differentiable rendering and consistency constraint optimization; and constructing a space-time consistency enhanced wiring information graph by taking the space coordinates as anchor points, carrying out multi-scale anomaly detection by adopting a self-supervised contrast learning model, and generating a defect significance graph and a diagnosis result. According to the method, the problems of poor identification stability, low cross-modal registration precision, difficulty in unknown defect detection and the like in a complex environment are effectively solved, and the automation degree and reliability of wiring state identification are remarkably improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of power grids, and particularly relates to a joint terminal box wiring state recognition and diagnosis method and device. BACKGROUND

[0002] In the field of electrical equipment maintenance and quality control, pattern recognition is performed on the image of a joint terminal box to determine the internal wiring state, which is a key safety technology. In the prior art, an automatic pattern recognition system is usually used, which obtains a two-dimensional digital image of the terminal box through an image acquisition device, and applies computer vision methods for feature extraction and classification to realize recognition and state diagnosis of terminals, cables and other components. To improve reliability, some improved schemes also attempt to introduce three-dimensional point cloud data as auxiliary features, and fuse them with two-dimensional visual information.

[0003] However, these prior art solutions still have significant defects in actual industrial scenarios. First, the performance of the existing automatic pattern recognition system is poor in stability when facing changes in the visual domain. Under the variable lighting, shadow and viewing angle conditions in the industrial field, the key visual patterns used for classification and recognition will be severely distorted, directly leading to serious confusion in feature matching and classification decision of the pattern recognition algorithm or missing detection of specific targets. Second, although three-dimensional data is introduced, how to accurately match the semantic objects in the two-dimensional image with the discrete three-dimensional feature points is still a technical bottleneck, resulting in ineffective collaboration of multi-modal information. In addition, after the existing pattern recognition model is deployed, its classification performance will decrease sharply when the imaging conditions change, showing insufficient adaptability to new visual domains. At the same time, for defect recognition, most systems rely on training on a large-scale image sample library pre-labeled with various known defects. This strong dependence on training samples results in limited ability of the system to recognize unknown or occasional defect patterns that do not appear in the training set.

[0004] Therefore, there is an urgent need for a joint terminal box wiring state recognition and diagnosis method under variable working conditions to improve the robustness of wiring state recognition and the ability to recognize unknown wiring defects under complex working conditions. SUMMARY

[0005] The joint terminal box wiring state recognition and diagnosis method provided by the application solves the problems of poor stability of wiring state recognition, low cross-modal registration accuracy and difficulty in detecting unknown wiring defects under complex field working conditions by obtaining two-dimensional images and multispectral fusion point cloud data of the joint terminal box and its surrounding area, combining cross-domain feature decoupling and attention weighted semantic representation, three-dimensional geometric spectral field modeling based on implicit neural representation, and self-supervised contrast learning for anomaly detection, and completing wiring component recognition, spatial registration and wiring state diagnosis in a unified processing flow.

[0006] In a first aspect, the present application provides a joint terminal box wiring state recognition and diagnosis method, comprising: Obtaining multi-view two-dimensional image data of a joint terminal box and a surrounding area of the joint terminal box, and obtaining multispectral fused point cloud data that is spatially registered with the multi-view two-dimensional image data; Feature encoding is performed on the two-dimensional image, domain-invariant semantic feature components and domain-related style feature components are decoupled in a latent space based on a cross-domain feature decoupling mechanism, domain-related information in the semantic features is inhibited based on an adversarial constraint and an orthogonal constraint, spatial attention and channel attention are applied to intermediate feature maps at each level obtained from the domain-invariant semantic feature components in a hierarchical feature extraction network, and a semantic segmentation result and a two-dimensional topological semantic graph of each wiring assembly of the joint terminal box are generated; The semantic segmentation result and the multispectral fused point cloud are input into a pre-trained implicit neural representation model, a three-dimensional geometric spectral field of the joint terminal box is established in the implicit neural representation model, a rendering feature map is generated based on the three-dimensional geometric spectral field through differentiable rendering under a preset virtual camera pose, and three-dimensional spatial coordinates of each wiring assembly and a connection are obtained according to a semantic correspondence and a structural correspondence between the rendering feature map, the two-dimensional image and the two-dimensional topological semantic graph; Taking the three-dimensional spatial coordinates as anchor points, wiring area image blocks are extracted and registered from multi-time and multi-view images, a spatiotemporal consistency enhanced wiring information graph is constructed, a feature encoding network trained based on normal samples through self-supervised contrastive learning is used to perform multi-scale feature extraction and abnormality evaluation on the wiring information graph, and a wiring defect saliency map and a wiring state diagnosis result are generated.

[0007] By employing the above-described scheme, the joint junction box wiring status recognition and diagnosis system of this invention, through the cross-domain feature decoupling mechanism and dual attention weighting in the semantic representation module, decomposes image features in the latent space into domain-invariant semantic features and domain-related style features, and applies adversarial and orthogonal constraints. Combined with spatial / channel attention for adaptive fusion, it effectively overcomes the interference of ambient lighting, shadows, and viewing angle changes, achieving accurate and stable semantic segmentation results and two-dimensional topological semantic maps for wiring components under varying operating conditions, significantly improving the system's robustness in complex environments. Furthermore, through registration optimization driven by differentiable rendering, a three-dimensional geometric spectral field is constructed based on the two-dimensional topological semantic map and multispectral fusion point cloud. Spatial coordinates are iteratively optimized using cross-modal semantic and structural consistency loss functions, overcoming the technical bottleneck of effectively associating two-dimensional semantics and three-dimensional geometric features, and achieving high-precision spatial positioning at the connection point. Furthermore, the diagnostic module employs self-supervised contrastive learning for anomaly detection. It trains a feature encoding network using normal samples and achieves keen identification of unknown defects and minor faults through multi-scale feature extraction and anomaly scoring. This eliminates reliance on pre-labeled defect samples and significantly enhances the generalization ability for unknown faults. Moreover, through a multi-module collaborative closed-loop system architecture, data acquisition, semantic understanding, 3D registration, and diagnostic analysis are integrated into a unified process, forming a complete technical closed loop from multimodal perception to intelligent diagnosis. This comprehensively improves the automation, accuracy, and reliability of the joint junction box wiring status identification.

[0008] In some embodiments of the present invention, acquiring multi-view two-dimensional images of the junction box and its surrounding area, and acquiring multispectral fused point cloud data spatially registered with the multi-view two-dimensional image data, includes: A 3D laser scanner is used to collect 3D point cloud data of the junction box and its surrounding area. A multispectral camera is used to simultaneously collect multispectral images covering the visible and infrared bands. Based on a joint calibration model established by preset sensor intrinsic and extrinsic parameters, the multi-view multispectral images and the 3D point cloud data are spatially registered and aligned in a unified coordinate system to generate the multispectral fused point cloud.

[0009] Preferably, the acquisition process of the multispectral fusion point cloud is as follows: a three-dimensional laser scanner is used to acquire semantic representation data of the three-dimensional point cloud wiring area of ​​the junction box and its surrounding environment, and a multispectral camera is used to simultaneously acquire spectral channel images covering the visible light and infrared bands. The spectral channel images and the three-dimensional point cloud data are spatially registered and aligned using a preset sensor intrinsic and extrinsic parameter joint calibration model. The pixel-level spectral information of each spectral channel image is assigned to the corresponding three-dimensional point in space, generating multispectral fusion point cloud data in which each three-dimensional coordinate point carries multidimensional spectral information.

[0010] In some embodiments of the present invention, the process of decoupling feature encoding into domain-invariant semantic feature components and domain-related style feature components based on the cross-domain feature decoupling mechanism includes: Construct a feature extraction network that includes an encoder, a semantic feature branch, and a style feature branch, and connect a domain classifier to the output of the semantic feature branch; The domain classifier outputs classification results based on the environmental domain labels and calculates the classification loss. During the training process of the encoder, the encoder updates its parameters based on an adversarial training strategy. The direction of the encoder's parameter update is opposite to the direction of the parameter update that reduces the classification loss. The features output by the encoder are divided into domain-invariant semantic feature components and domain-related style feature components.

[0011] Preferably, the process of decoupling feature encoding into domain-invariant semantic feature components and domain-dependent style feature components based on the cross-domain feature decoupling mechanism specifically involves: inputting the two-dimensional image into a deep convolutional neural network encoder to extract initial features containing multi-level feature information from the bottom layer to the top layer, wherein the initial features include the low-level visual features of the image and the local shape and structural information of the wiring components; the cross-domain feature decoupling mechanism explicitly decomposes the initial features in the latent space into domain-invariant semantic feature components representing the inherent properties of the wiring components and domain-dependent style feature components representing the changes in the external environment of the wiring components, and enhances the independence of the feature components by applying orthogonality constraints and adversarial loss. The adversarial loss is achieved by connecting an auxiliary domain classifier to the semantic feature component output of the encoder. This domain classifier is used to determine which environmental domain the feature originates from. The training objective of the encoder is to generate a classification loss that maximizes the domain classifier, which is an adversarial loss. The orthogonality constraint is specifically achieved by calculating the cosine similarity between the domain-invariant semantic feature component and the domain-related style feature component vector, and adding the absolute value as a loss term to the total loss function to minimize the correlation between the two subspaces. The domain-invariant semantic feature component is input to the deep convolutional neural network decoder for upsampling and pixel-level classification to generate an intermediate feature map that eliminates the influence of differences in environment and acquisition equipment.

[0012] In some embodiments of the present invention, training the encoder further includes: A mutual information estimate is calculated between the semantic feature components and the style feature components, and a mutual information loss is constructed to penalize the information redundancy between them. Furthermore, a reconstruction loss is constructed based on the reconstruction error between the image reconstructed by the decoder and the original two-dimensional image. The encoder and decoder are then optimized by combining the mutual information loss and the reconstruction loss to obtain an encoding result in which the domain-related information components in the semantic features are weakened.

[0013] In some embodiments of the present invention, the step of applying spatial attention and channel attention to the intermediate feature maps of each level obtained from the domain-invariant semantic feature components in the hierarchical feature extraction network, and combining spatial attention and channel attention to generate semantic segmentation results and two-dimensional topological semantic maps of each wiring component of the junction box, includes: The intermediate feature maps output by each layer of the decoder in the hierarchical feature extraction network are weighted by a channel attention network. The channel attention network obtains the channel weight vector through global pooling and then weights each channel. A two-dimensional attention mask is generated in the spatial dimension of the feature map by a spatial attention network, and each spatial position of the intermediate feature map is weighted position by position to obtain an attention-weighted feature map. The feature response value of the wiring component area in the attention-weighted feature map is higher than that of the background area.

[0014] Preferably, the semantic representation generation process of the wiring region specifically involves: in the hierarchical feature extraction network, a spatial and channel dual attention mechanism is applied to the intermediate feature map output at each level for progressive adaptive weighting; the spatial and channel dual attention mechanism includes a channel attention network and a spatial attention network, wherein the channel attention network captures the dependencies between channels through global pooling operations to generate channel weight vectors, and the spatial attention network generates a two-dimensional attention mask in the spatial dimension of the feature map; by applying the channel weight vector and the two-dimensional attention mask together to the corresponding intermediate feature map for weighting, the low-level detailed information and high-level semantic information are effectively fused to generate a semantic representation of the wiring region that can describe the type, location, and interrelationship of the wiring components.

[0015] In some embodiments of the present invention, establishing the three-dimensional geometric spectral field of the joint junction box in the implicit neural representation model includes: An implicit neural representation model with a multilayer perceptron as the main structure is constructed. The three-dimensional spatial coordinates after position encoding and the feature vectors of the corresponding virtual observation view direction are used as inputs. The geometric density and multispectral feature values ​​of the corresponding spatial points are output. The rendered feature map is generated by differentiable rendering based on the geometric density and multispectral feature values.

[0016] Preferably, the construction process of the three-dimensional geometric spectral field is as follows: by segmenting the semantic representation of the wiring region through instances, each independent wiring component is identified, each component is abstracted into a semantic node carrying category and location attributes, and edges are established based on the spatial proximity relationship and connection logic between components to construct a two-dimensional topological semantic graph describing the two-dimensional connection relationship between components; the two-dimensional region represented by each semantic node in the two-dimensional topological semantic graph is spatially mapped and associated with the multispectral fusion point cloud to construct a supervised dataset containing three-dimensional coordinate points and corresponding real geometric density, multispectral values ​​and semantic categories; the implicit neural representation model is trained using the supervised dataset to enable the implicit neural representation model to learn a continuous mapping function from arbitrary three-dimensional spatial coordinates and observation viewpoint direction to geometric density and multispectral values, and finally form a three-dimensional geometric spectral field. The three-dimensional geometric spectral field serves as a unified function representation and includes the three-dimensional geometric structure of the wiring components and multispectral information reflecting the physical properties of the materials.

[0017] In some embodiments of the present invention, the training method of the pre-trained implicit neural representation model includes: The gradient norm of the geometric density field in the three-dimensional geometric spectral field in the neighborhood of the joint junction box surface is calculated, and the surface normal gradient constraint loss is constructed based on the gradient norm. For three-dimensional spatial points belonging to the same semantic category, calculate the difference between the corresponding predicted multispectral feature values ​​and construct the semantic spectrum internal consistency loss; During the parameter update process, the weighted sum of the reconstruction loss, the surface normal gradient constraint loss, and the semantic spectrum internal consistency loss is used as the total loss to iteratively update the parameters of the implicit neural representation model.

[0018] In some embodiments of the present invention, the training method of the pre-trained implicit neural representation model further includes: The difference at the pixel level between the physical data channels in the multi-channel feature map obtained by differentiable rendering of the three-dimensional geometric spectral field and the corresponding physical data channels in the original multispectral image after calibration and alignment is calculated, and a physical property consistency loss is constructed based on the difference. During the parameter update process, the weighted sum of the physical property consistency loss, semantic consistency loss, and structural consistency loss is used as the optimization objective to iteratively update the parameters of the implicit neural representation model.

[0019] Preferably, the process of acquiring the multi-channel feature map is as follows: setting an initial virtual camera pose corresponding to the two-dimensional image, starting from the virtual camera viewpoint, emitting sampling rays to the pixels of the multi-channel feature map to be generated, passing through the three-dimensional geometric spectral field, and performing differentiable integral operations on the geometric density and multispectral feature values ​​of the sampling points in the field along the ray path to calculate the final multi-channel feature value of each pixel, and assigning and arranging the multi-channel feature values ​​according to the original two-dimensional pixel coordinates of the pixel on the feature map to be generated, thereby constructing a complete multi-channel feature map with a preset spatial resolution and number of channels.

[0020] Preferably, the multi-channel feature map is compared with the semantic representation of the connection area. The semantic consistency loss is defined by calculating the difference in pixel-level semantic information between the multi-channel feature map and the connection area. Combined with the two-dimensional topological semantic map, the structural consistency loss is defined by measuring the consistency between the component layout in the rendering result and the connection relationship in the topological map. Together, they constitute the cross-modal semantic and structural consistency loss function. The cross-modal semantic and structural consistency loss function is used as the optimization objective. The parameters of the three-dimensional geometric spectral field and the virtual camera pose used for rendering are iteratively adjusted through the backpropagation algorithm until the cross-modal semantic and structural consistency loss function converges to the minimum value. The three-dimensional center point and feature point of the geometric region defined semantically as the connection point are directly queried and extracted from the registered three-dimensional geometric spectral field as the spatial coordinates of the connection point.

[0021] Preferably, for the two-dimensional image sequence acquired from different viewpoints and time points, a photometric correction algorithm is applied to eliminate pixel brightness and chromaticity inconsistencies; using the spatial coordinates of the connection point as three-dimensional anchor points, the three-dimensional spatial region is back-projected onto each photometrically corrected two-dimensional image to extract multi-view image blocks corresponding to the same physical connection area; a projection transformation model based on camera pose uses the camera parameters corresponding to each view image block to calculate the projection transformation matrix required to transform the image block from the original acquisition viewpoint to a unified, standardized two-dimensional orthophoto plane, and applies the projection transformation matrix to perform geometric correction and precise alignment of each image block; during the fusion process, a weighted average and pixel quality metric-based selection strategy is used to suppress noise and enhance defect details in the overlapping areas of the aligned image blocks, generating an enhanced connection information map that exhibits spatiotemporal consistency in texture, color, and structure.

[0022] In some embodiments of the present invention, the step of using a feature encoding network trained by self-supervised contrastive learning based on normal samples to perform multi-scale feature extraction and anomaly assessment on the wiring information diagram, and generating a wiring defect saliency map and wiring status diagnosis results includes: In the pre-training phase of the implicit neural representation model, normal wiring information diagrams without wiring defects are used as training samples. For each normal wiring information diagram, a data augmentation operation is performed. The data augmentation operation includes at least one or more of random cropping, random flipping, color perturbation, and blurring. Positive sample items are constructed based on sample pairs formed by the same original wiring information diagram in the augmented image, and negative sample items are constructed based on sample pairs formed by different original wiring information diagrams. The contrastive learning loss is calculated based on the feature distance between the positive sample items and the negative sample items, and the feature encoding network parameters are updated. During the diagnostic phase, the spatiotemporal consistency enhanced wiring information map is input into the feature encoding network to obtain multi-scale feature representations. Based on the similarity or distance between the multi-scale features and the normal sample feature set obtained in the pre-training phase, the anomaly score of each spatial location is calculated to construct a wiring defect saliency map. According to the comparison results of the anomaly score of each wiring region in the wiring defect saliency map with the preset threshold, the wiring regions are classified to obtain the wiring status diagnostic results.

[0023] Preferably, the process of using a feature encoding network trained on normal samples through self-supervised contrastive learning to perform multi-scale feature extraction and anomaly assessment on the wiring information map, generating a wiring defect saliency map and wiring status diagnosis results, specifically involves: using defect-free normal wiring information maps as training samples, applying data augmentation transformations to each sample to generate positive and negative sample pairs, and performing self-supervised pre-training based on a contrastive learning loss function to obtain an encoder capable of mapping the features of normal wiring states to a specific latent space; during recognition, the spatiotemporal consistency-enhanced wiring information map to be diagnosed is input into the multi-scale... A feature extraction network is used to obtain feature maps representing different receptive fields and resolutions. The difference between each level's feature map and the normal feature space learned during pre-training is measured pixel-wise and region-wise, and the corresponding anomaly score map is calculated. All anomaly score maps at all scales are upsampled to the same spatial resolution as the input information map, and then these resolution-consistent anomaly score maps are weighted and summed pixel-wise. All scales of anomaly score maps are integrated to generate a defect saliency map. Thresholds are set on the defect saliency map to locate and segment defect regions of different sizes, obtaining the identification and diagnosis results.

[0024] Compared with existing technologies, the advantages of this invention lie in its use of multispectral fusion acquisition technology. This technology leverages the collaborative work of a 3D laser scanner and a common-path multispectral camera array, combined with a joint calibration model based on a multispectral feature calibration board, to achieve simultaneous acquisition and precise alignment of multidimensional physical information such as visible light, near-infrared, and thermal infrared. This provides a high-quality multimodal data foundation with both geometric accuracy and physical properties for subsequent analysis. Furthermore, through cross-domain feature decoupling and a dual attention weighting mechanism, image features are decomposed into domain-invariant semantic features and domain-related style features in the latent space. Adversarial loss and orthogonality constraints are applied, and adaptive fusion is performed using spatial / channel attention. This effectively overcomes the interference of ambient lighting, shadows, and viewing angle changes, enabling the generation of accurate and stable semantic segmentation results and two-dimensional topological semantic maps for wiring components under varying operating conditions. This significantly improves the system's robustness in complex environments. By employing self-supervised contrastive learning and multi-scale anomaly detection, and utilizing a feature encoding network trained on normal samples, combined with a feature pyramid structure for multi-scale feature extraction and anomaly scoring, a defect saliency map is generated, enabling keen identification of unknown defects and minor faults. Simultaneously, a photometrically corrected multi-view sequence fusion technique is used to construct a spatiotemporally consistent enhanced wiring information map using three-dimensional spatial coordinates as anchor points, significantly improving the generalization ability and diagnostic reliability for unknown faults. Through the collaborative operation of these multiple modules, the system forms a complete technical closed loop from multimodal data perception, robust semantic understanding, accurate spatial registration to intelligent diagnostic analysis. In the joint junction box status monitoring of complex scenarios such as high-speed railways, it comprehensively improves the automation, accuracy, and engineering practical value of identification and diagnosis.

[0025] A second aspect of the present invention provides a combined junction box wiring status identification and diagnostic system, comprising: The data acquisition module is used to acquire multi-view two-dimensional image data of the junction box and the surrounding area of ​​the junction box; and to acquire multispectral fused point cloud data spatially registered with the multi-view two-dimensional image data. The semantic representation module, connected to the data acquisition module, is used to perform feature encoding on the two-dimensional image. Based on the cross-domain feature decoupling mechanism, the feature encoding is decoupled into domain-invariant semantic feature components and domain-related style feature components in the latent space. Based on adversarial constraints and orthogonal constraints, the domain-related information in the semantic features is suppressed. In the hierarchical feature extraction network, spatial attention and channel attention are applied to the intermediate feature maps of each level obtained from the domain-invariant semantic feature components to generate the semantic segmentation results of each wiring component of the joint junction box and a two-dimensional topological semantic map. A 3D registration module, connected to the semantic representation module and the data acquisition module, is used to input the semantic segmentation result and the multispectral fusion point cloud into a pre-trained implicit neural representation model, and to establish a 3D geometric spectral field of the joint junction box in the implicit neural representation model; under a preset virtual camera pose, a rendering feature map is generated based on the 3D geometric spectral field through differentiable rendering; and the 3D spatial coordinates of each wiring component and connection point are obtained according to the semantic and structural correspondences between the rendering feature map, the 2D image, and the 2D topological semantic map. The multi-level diagnostic module, connected to the three-dimensional registration module and the data acquisition module, is used to extract and register wiring area image blocks from multi-time and multi-view images using the three-dimensional spatial coordinates as anchor points, construct a spatiotemporal consistency enhanced wiring information map, and use a feature encoding network trained by self-supervised contrastive learning based on normal samples to perform multi-scale feature extraction and anomaly assessment on the wiring information map, generating a wiring defect saliency map and wiring status diagnostic results.

[0026] A third aspect of the present invention provides a combined junction box wiring status identification and diagnosis device, characterized in that the device includes a computer device, the computer device includes a processor and a memory, the processor stores computer instructions, and when the computer instructions are executed, the device implements the combined junction box wiring status identification and diagnosis system.

[0027] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the text, or may be learned by practice of the invention. The objects and other advantages of the invention will become apparent from the description and the accompanying drawings.

[0028] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0029] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0030] In the attached diagram: Figure 1 This is a flowchart illustrating a method for identifying and diagnosing the wiring status of a junction box, as provided in an embodiment of the present invention.

[0031] Figure 2 This is a schematic diagram of a combined junction box wiring status identification and diagnosis system module provided in an embodiment of the present invention.

[0032] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0033] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0034] The following detailed description is exemplary and intended to provide further detailed explanation of the invention. Unless otherwise specified, all technical terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used in this invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention.

[0035] Figure 1 This is a flowchart illustrating a combined junction box wiring status identification and diagnosis system according to an embodiment of the present invention.

[0036] Example 1, such as Figure 1 As shown, the present invention provides a method for identifying and diagnosing the wiring status of a junction box, the method comprising the following steps: S1. Acquire multi-view two-dimensional image data of the junction box and the surrounding area of ​​the junction box; acquire multispectral fused point cloud data spatially registered with the multi-view two-dimensional image data; S2. The two-dimensional image is feature encoded. Based on the cross-domain feature decoupling mechanism, the feature encoding is decoupled into domain-invariant semantic feature components and domain-related style feature components in the latent space. Based on adversarial constraints and orthogonal constraints, the domain-related information in the semantic features is suppressed. In the hierarchical feature extraction network, spatial attention and channel attention are applied to the intermediate feature maps of each level obtained from the domain-invariant semantic feature components to generate the semantic segmentation results of each wiring component of the joint junction box and the two-dimensional topological semantic map. S3. Input the semantic segmentation result and the multispectral fusion point cloud into a pre-trained implicit neural representation model, and establish a three-dimensional geometric spectral field of the joint junction box in the implicit neural representation model; under the preset virtual camera pose, generate a rendering feature map based on the three-dimensional geometric spectral field through differentiable rendering; obtain the three-dimensional spatial coordinates of each wiring component and connection point according to the semantic correspondence and structural correspondence between the rendering feature map, the two-dimensional image and the two-dimensional topological semantic map; S4. Using the three-dimensional spatial coordinates as anchor points, extract and register wiring area image blocks from multi-time and multi-view images to construct a spatiotemporal consistency enhanced wiring information map. Utilize a feature encoding network trained based on normal samples through self-supervised contrastive learning to perform multi-scale feature extraction and anomaly assessment on the wiring information map, generating a wiring defect saliency map and wiring status diagnosis results.

[0037] By employing the above-described scheme, the joint junction box wiring status recognition and diagnosis system of this invention, through the cross-domain feature decoupling mechanism and dual attention weighting in the semantic representation module, decomposes image features in the latent space into domain-invariant semantic features and domain-related style features, and applies adversarial and orthogonal constraints. Combined with spatial / channel attention for adaptive fusion, it effectively overcomes the interference of ambient lighting, shadows, and viewing angle changes, achieving accurate and stable semantic segmentation results and two-dimensional topological semantic maps for wiring components under varying operating conditions, significantly improving the system's robustness in complex environments. Furthermore, through registration optimization driven by differentiable rendering, a three-dimensional geometric spectral field is constructed based on the two-dimensional topological semantic map and multispectral fusion point cloud. Spatial coordinates are iteratively optimized using cross-modal semantic and structural consistency loss functions, overcoming the technical bottleneck of effectively associating two-dimensional semantics and three-dimensional geometric features, and achieving high-precision spatial positioning at the connection point. Furthermore, the diagnostic module employs self-supervised contrastive learning for anomaly detection. It trains a feature encoding network using normal samples and achieves keen identification of unknown defects and minor faults through multi-scale feature extraction and anomaly scoring. This eliminates reliance on pre-labeled defect samples and significantly enhances the generalization ability for unknown faults. Moreover, through a multi-module collaborative closed-loop system architecture, data acquisition, semantic understanding, 3D registration, and diagnostic analysis are integrated into a unified process, forming a complete technical closed loop from multimodal perception to intelligent diagnosis. This comprehensively improves the automation, accuracy, and reliability of the joint junction box wiring status identification.

[0038] In some embodiments of the present invention, in step S1, acquiring multi-view two-dimensional images of the junction box and the surrounding area of ​​the junction box, and acquiring multispectral fused point cloud data spatially registered with the multi-view two-dimensional image data, includes: A 3D laser scanner is used to collect 3D point cloud data of the junction box and its surrounding area. A multispectral camera is used to simultaneously collect multispectral images covering the visible and infrared bands. Based on a joint calibration model established by preset sensor intrinsic and extrinsic parameters, the multi-view multispectral images and the 3D point cloud data are spatially registered and aligned in a unified coordinate system to generate the multispectral fused point cloud.

[0039] In this embodiment, the system is deployed as an automated track inspection robot capable of autonomously navigating the railway tracks. Its core workflow begins with the data acquisition module performing comprehensive, multimodal, high-precision data acquisition of the target junction box. This module precisely integrates an industrial-grade high-resolution 3D laser scanner and a specially designed, rigorously radiometrically calibrated multispectral imaging unit. The imaging unit is a common-path camera array capable of simultaneously acquiring images covering multiple spectral channels, including visible light, near-infrared, and long-wave thermal infrared. When the inspection robot travels to the optimal observation position of the target junction box according to the preset task path, the data acquisition process is automatically triggered. The laser scanner scans the junction box, acquiring dense 3D point cloud data of the junction box itself, external cables, fixed supports, and even the surrounding environment, depicting its macroscopic and microscopic geometric shape. Simultaneously, the multispectral camera array simultaneously exposes, capturing a sequence of multispectral images within the same field of view.

[0040] Furthermore, during the offline phase, the system has established a rigid body transformation model through a precise joint calibration process of sensor intrinsic and extrinsic parameters. This process uses a specially designed calibration board with significant characteristics across different spectra to accurately calculate the rotation and translation matrices between the laser scanner coordinate system and the coordinate systems of each sensor in the multispectral camera array. During online operation, the model can accurately project every pixel of each frame of multispectral image into the 3D point cloud space in real time, achieving precise registration between pixels and 3D points in space. This process is not a simple information overlay but employs a novel multispectral fusion point cloud data structure. In this data structure, the data attributes of any 3D point in space, such as the point representing the top surface of a fastening screw, are greatly enriched: it not only possesses (X, Y, Z) coordinates describing its spatial position and (R, G, B) color values, but is also endowed with near-infrared spectral reflectance reflecting subtle material differences and early surface oxidation states, as well as thermal infrared radiation values ​​directly revealing the temperature rise caused by current flow or poor contact.

[0041] Multispectral fusion point clouds combine geometric and physical information, providing richer spatial data than traditional data for subsequent semantic understanding and diagnosis, which helps to improve the probability and accuracy of fault detection.

[0042] In some embodiments of the present invention, in step S2, the process of decoupling feature encoding into domain-invariant semantic feature components and domain-related style feature components based on the cross-domain feature decoupling mechanism includes: Construct a feature extraction network that includes an encoder, a semantic feature branch, and a style feature branch, and connect a domain classifier to the output of the semantic feature branch; The domain classifier outputs classification results based on the environmental domain labels and calculates the classification loss. During the training process of the encoder, the encoder updates its parameters based on an adversarial training strategy. The direction of the encoder's parameter update is opposite to the direction of the parameter update that reduces the classification loss. The features output by the encoder are divided into domain-invariant semantic feature components and domain-related style feature components.

[0043] Preferably, the process of decoupling feature encoding into domain-invariant semantic feature components and domain-dependent style feature components based on the cross-domain feature decoupling mechanism specifically involves: inputting the two-dimensional image into a deep convolutional neural network encoder to extract initial features containing multi-level feature information from the bottom layer to the top layer, wherein the initial features include the low-level visual features of the image and the local shape and structural information of the wiring components; the cross-domain feature decoupling mechanism explicitly decomposes the initial features in the latent space into domain-invariant semantic feature components representing the inherent properties of the wiring components and domain-dependent style feature components representing the changes in the external environment of the wiring components, and enhances the independence of the feature components by applying orthogonality constraints and adversarial loss. The adversarial loss is achieved by connecting an auxiliary domain classifier to the semantic feature component output of the encoder. This domain classifier is used to determine which environmental domain the feature originates from. The training objective of the encoder is to generate a classification loss that maximizes the domain classifier, which is an adversarial loss. The orthogonality constraint is specifically achieved by calculating the cosine similarity between the domain-invariant semantic feature component and the domain-related style feature component vector, and adding the absolute value as a loss term to the total loss function to minimize the correlation between the two subspaces. The domain-invariant semantic feature component is input to the deep convolutional neural network decoder for upsampling and pixel-level classification to generate an intermediate feature map that eliminates the influence of differences in environment and acquisition equipment.

[0044] In some embodiments of the present invention, training the encoder further includes: A mutual information estimate is calculated between the semantic feature components and the style feature components, and a mutual information loss is constructed to penalize the information redundancy between them. Furthermore, a reconstruction loss is constructed based on the reconstruction error between the image reconstructed by the decoder and the original two-dimensional image. The encoder and decoder are then optimized by combining the mutual information loss and the reconstruction loss to obtain an encoding result in which the domain-related information components in the semantic features are weakened.

[0045] In some embodiments of the present invention, in step S2, applying spatial attention and channel attention to the intermediate feature maps of each level obtained from the domain-invariant semantic feature components in the hierarchical feature extraction network, and combining spatial attention and channel attention to generate the semantic segmentation results and two-dimensional topological semantic maps of each wiring component of the joint junction box, includes: The intermediate feature maps output by each layer of the decoder in the hierarchical feature extraction network are weighted by a channel attention network. The channel attention network obtains the channel weight vector through global pooling and then weights each channel. A two-dimensional attention mask is generated in the spatial dimension of the feature map by a spatial attention network, and each spatial position of the intermediate feature map is weighted position by position to obtain an attention-weighted feature map. The feature response value of the wiring component area in the attention-weighted feature map is higher than that of the background area.

[0046] Preferably, the semantic representation generation process of the wiring region specifically involves: in the hierarchical feature extraction network, a spatial and channel dual attention mechanism is applied to the intermediate feature map output at each level for progressive adaptive weighting; the spatial and channel dual attention mechanism includes a channel attention network and a spatial attention network, wherein the channel attention network captures the dependencies between channels through global pooling operations to generate channel weight vectors, and the spatial attention network generates a two-dimensional attention mask in the spatial dimension of the feature map; by applying the channel weight vector and the two-dimensional attention mask together to the corresponding intermediate feature map for weighting, the low-level detailed information and high-level semantic information are effectively fused to generate a semantic representation of the wiring region that can describe the type, location, and interrelationship of the wiring components.

[0047] Specifically, after obtaining multimodal data containing high-definition visible light images in step S1, the system initiates step S2 to perform semantic representation processing and parse the image content. The original two-dimensional visible light image is fed into an encoder based on a deep convolutional neural network (in this embodiment, an architecture based on ResNet-50 and combined with a feature pyramid network) to extract multi-scale initial features. These initial features are a mixture of low-level visual elements, intermediate-level component local shapes, and high-level component structure and spatial arrangement information. However, ambient conditions (such as midday highlights, evening shadows, lens dust, or raindrops) can easily cause local blurring, which can interfere with the initial features and lead to a decrease in recognition accuracy.

[0048] This mechanism introduces an adversarial learning framework into the latent feature space of the neural network. It forcibly decomposes the extracted mixed initial features into two mathematically orthogonal and informationally independent components. One is a domain-invariant semantic feature component, which carries the inherent, environmentally unchanging properties of the wiring components, such as shape, structure, and topological relationships. The other is a domain-dependent style feature component, which specifically captures all changes caused by the external environment (light intensity and angle, shooting perspective, weather conditions) and the state of the acquisition device itself (lens dirt, sensor noise). To achieve this complete decoupling, the system designs an auxiliary domain classifier, whose task is to attempt to determine the shooting environment of the image based on the feature components. The backbone network aims to accurately segment the components while generating semantic features that the domain classifier cannot determine, thereby forcing the semantic feature components to eliminate all environment-related information through this deceptive behavior. At the same time, the applied orthogonality constraint ensures that the semantic and style subspaces are completely independent.

[0049] The cross-domain feature decoupling mechanism also includes an encoder, a semantic feature branch, a style feature branch, and a decoder for reconstructing the two-dimensional image. The semantic representation module is also used to calculate the mutual information loss between the outputs of the semantic feature branch and the style feature branch, and combine it with the image reconstruction loss based on the decoder output to jointly optimize the encoder, so as to force the minimization of the information correlation between the semantic features and the style features.

[0050] Specifically, to achieve more thorough decoupling, this embodiment introduces a dual constraint on top of the adversarial loss. First, a mutual information estimator is introduced to evaluate the information correlation between the semantic feature vector and the style feature vector in real time, and the negative of the mutual information is fed back to the encoder as an additional loss (i.e., mutual information minimization loss), forcing the two subspaces to be independent at the information theory level. Second, the decoupled semantic features and style features are re-merged and fed into a decoder symmetrical to the encoder, forcing it to reconstruct the original input image with high accuracy. This reconstruction loss (e.g., L1 or L2 loss) ensures that the decoupling process does not lose any information in the original image, but merely redistributes it to two orthogonal subspaces.

[0051] During training, the total loss function Defined as the weighted sum of all losses: in For adversarial loss (e.g., cross-entropy loss of domain classifiers). Image reconstruction loss (e.g., L1 loss). To minimize the loss of mutual information, The orthogonality constraint loss between semantic features and style features (e.g., the L2 norm of their inner product).

[0052] These are preset weighting coefficients for various losses, used to balance the decoupling effect and reconstruction accuracy.

[0053] By minimizing mutual information and ensuring reconstruction consistency, the system ensures the high purity of semantic features, making them truly domain-invariant and significantly improving the semantic segmentation accuracy of the model under harsh conditions such as extreme lighting, shadows, or lens contamination.

[0054] Domain-invariant semantic feature components, unaffected by external interference, are fed into the decoder part of the network for upsampling and pixel-level classification prediction, thereby generating clear and accurate intermediate feature maps.

[0055] In the intermediate feature map, each pixel in the image is precisely assigned a semantic label, achieving stable and robust semantic understanding of complex outdoor scenes at the pixel level, which is beneficial for subsequent weighted processing of the feature map.

[0056] Simultaneously, a dual spatial and channel attention mechanism is employed to perform hierarchical adaptive weighting on the aforementioned intermediate feature maps, generating a final, more information-rich semantic representation of the wiring region. In a hierarchical feature extraction network similar to U-Net, the feature map output by each level of the decoder passes through an attention module composed of channel attention and spatial attention networks (which can be connected in series or parallel). The channel attention network analyzes the importance of different feature channels for identifying specific components through global average pooling and max pooling operations (e.g., when distinguishing wires, color and diameter-related channels have higher weights than texture channels) and assigns higher weights to key channels. The spatial attention network operates in the spatial dimension of the feature map, generating a two-dimensional attention mask, which concentrates computational resources on key regions (such as wire-terminal crimping points, screw-nut meshing points), while ignoring flat areas of the background or components. Applying these two attention mechanisms to feature maps at different levels enables efficient and selective fusion of shallow detail information and deep high-level semantic information.

[0057] The semantic representation of the wiring region generated by the dual attention mechanism has high information density, depicts the category, pixel-level outline and location of the components, and encodes the spatial adjacency and logical connection relationships between the components for subsequent topology graph construction.

[0058] Further, after semantic representation processing of the image in step S2, the system enters the 3D registration process. First, the system performs instance segmentation on the semantic representation map generated in the previous step, identifying each independent wiring component instance; then, each component instance is abstracted into a semantic node carrying a unique ID, category, two-dimensional location, and shape attributes; based on the electrical engineering schematic diagram and the spatial proximity relationship between components, the system establishes connecting edges between these nodes; specifically, the system adopts a two-stage strategy: first, based on the distance of the semantic nodes in the image space (e.g., calculating the minimum distance of the instance segmentation mask) and a preset proximity threshold, edges representing spatial proximity are initially established; second, a preset electrical connection rule library is loaded (e.g., stipulating that "cable end" must be connected to "terminal crimping point"), and the initially established edges are filtered and corrected through a graph neural network or rule inference engine, removing adjacent edges that do not conform to electrical logic and filling in missing logical connecting edges, thereby constructing a two-dimensional topological semantic graph containing rich logical information. This constructs a two-dimensional topological semantic graph containing rich logical information; this graph is a logical blueprint of the internal connection relationship of the junction box and an abstract expression of its electrical function.

[0059] At the same time, with this blueprint, the system began to construct the three-dimensional geometric spectral field of the junction box; In some embodiments of the present invention, in step S3, establishing the three-dimensional geometric spectral field of the joint junction box in the implicit neural representation model includes: An implicit neural representation model with a multilayer perceptron as the main structure is constructed. The three-dimensional spatial coordinates after position encoding and the feature vectors of the corresponding virtual observation view direction are used as inputs. The geometric density and multispectral feature values ​​of the corresponding spatial points are output. The rendered feature map is generated by differentiable rendering based on the geometric density and multispectral feature values.

[0060] Preferably, the construction process of the three-dimensional geometric spectral field is as follows: by segmenting the semantic representation of the wiring region through instances, each independent wiring component is identified, each component is abstracted into a semantic node carrying category and location attributes, and edges are established based on the spatial proximity relationship and connection logic between components to construct a two-dimensional topological semantic graph describing the two-dimensional connection relationship between components; the two-dimensional region represented by each semantic node in the two-dimensional topological semantic graph is spatially mapped and associated with the multispectral fusion point cloud to construct a supervised dataset containing three-dimensional coordinate points and corresponding real geometric density, multispectral values ​​and semantic categories; the implicit neural representation model is trained using the supervised dataset to enable the implicit neural representation model to learn a continuous mapping function from arbitrary three-dimensional spatial coordinates and observation viewpoint direction to geometric density and multispectral values, and finally form a three-dimensional geometric spectral field. The three-dimensional geometric spectral field serves as a unified function representation and includes the three-dimensional geometric structure of the wiring components and multispectral information reflecting the physical properties of the materials.

[0061] Specifically, in this embodiment, the two-dimensional image region represented by each semantic node in the two-dimensional topological semantic graph is initially spatially mapped and associated with the previously acquired multispectral fused point cloud to construct a supervised dataset. Each data point in this dataset is a multidimensional vector containing the precise coordinates of a three-dimensional point, the geometric density value indicating whether the point is on the surface of an object, its complete multispectral reflectance / radiance values, and a precise semantic category label transmitted from the two-dimensional semantic graph. Using this supervised dataset, the system performs deep training on an implicit neural representation model. The goal of the training is to enable the model to learn a highly nonlinear, continuous mapping function from any three-dimensional spatial coordinates to the complete set of attributes of that point.

[0062] In some embodiments of the present invention, in step S3, when training the implicit neural representation model, in addition to optimizing based on the reconstruction loss between the rendered feature map and the two-dimensional image, the training method of the pre-trained implicit neural representation model includes: The gradient norm of the geometric density field in the three-dimensional geometric spectral field in the neighborhood of the joint junction box surface is calculated, and the surface normal gradient constraint loss is constructed based on the gradient norm. For three-dimensional spatial points belonging to the same semantic category, calculate the difference between the corresponding predicted multispectral feature values ​​and construct the semantic spectrum internal consistency loss; During the parameter update process, the weighted sum of the reconstruction loss, the surface normal gradient constraint loss, and the semantic spectrum internal consistency loss is used as the total loss to iteratively update the parameters of the implicit neural representation model.

[0063] In this embodiment, the three-dimensional geometric spectral field is an implicit neural representation model, and the three-dimensional registration module applies surface normal gradient constraint loss and semantic spectral internal consistency loss during the training process of constructing the three-dimensional geometric spectral field. The semantic spectral internal consistency loss is used to penalize the implicit neural representation model for predicting inconsistent spectral values ​​for three-dimensional spatial points with the same semantic category. The implicit neural representation model specifically adopts a multilayer perceptron architecture. The input three-dimensional spatial coordinates and two-dimensional observation viewpoint direction are first mapped to high-dimensional feature vectors through a position encoding function. The high-dimensional feature vectors of the coordinates are fed into a backbone network containing 8 fully connected layers (256 neurons per layer, using the ReLU activation function). The high-dimensional feature vector of the viewpoint direction is concatenated with the coordinate features after the 5th layer of the backbone network and input together into subsequent network layers. The network finally outputs the geometric density and multispectral value of the point.

[0064] Specifically, when training this implicit model, the system adds two physical prior-based regularization losses in addition to the standard reconstruction loss. First, there's the surface normal gradient constraint loss, which mandates that the gradient norm (i.e., rate of change) of the geometric density field in space is always equal to 1 in the region near the object's surface. This forces the model to learn clear, sharp object surfaces, rather than blurry, hazy transitions. Second, there's the semantic spectral internal consistency loss. In training batches, for all sampling points predicted by the model as having the same semantic label (e.g., "copper terminal"), the system calculates the variance of their predicted multispectral values ​​(e.g., near-infrared and thermal infrared values) and minimizes this variance as a loss term, forcing the model to learn the prior knowledge that "materials of the same type should have highly consistent physical properties."

[0065] The three-dimensional geometric spectral field constructed by this strategy is sharper geometrically and more physically consistent, significantly improving the realism of digital twins and the accuracy of subsequent rendering, while avoiding blur or artifacts that do not conform to physical laws.

[0066] After training, the model becomes a three-dimensional geometric spectral field. This field is not a static three-dimensional model composed of discrete meshes or point clouds, but a continuous and differentiable function. It can not only query the properties of any spatial point, but also render the geometry of the junction box at arbitrarily high resolution.

[0067] In some embodiments of the present invention, in step S3, the training method of the pre-trained implicit neural representation model further includes: The difference at the pixel level between the physical data channels in the multi-channel feature map obtained by differentiable rendering of the three-dimensional geometric spectral field and the corresponding physical data channels in the original multispectral image after calibration and alignment is calculated, and a physical property consistency loss is constructed based on the difference. During the parameter update process, the weighted sum of the physical property consistency loss, semantic consistency loss, and structural consistency loss is used as the optimization objective to iteratively update the parameters of the implicit neural representation model.

[0068] Specifically, to achieve pixel-level precise alignment between this 3D digital twin and the real 2D image, the system employs differentiable rendering technology. The system sets an initial virtual camera pose corresponding to the real 2D image acquisition pose. Starting from the virtual camera's viewpoint, a sampling ray is emitted along the line of sight for each pixel of the multi-channel feature map to be generated, passing through the 3D geometric spectral field. Along this ray's path, the system performs volume rendering integral calculations, weighting and accumulating the geometric density and multispectral feature values ​​of countless sampling points on the ray to calculate the final multi-channel feature value of the pixel. By arranging and combining the calculation results of all pixels, a rendered feature map is constructed that is completely identical to the real multispectral image in resolution and number of channels. At this point, the system quantifies the accuracy of the current registration by calculating the cross-modal semantic and structural consistency loss between this rendered map and the previously generated semantic representation of the wiring region. This loss function is carefully designed to include two parts: one is semantic consistency loss, which ensures that the rendered component category is positive by calculating the cross-entropy loss between the rendered map and the semantic representation map on pixel-level semantic labels. The first is the structural consistency loss, which combines a two-dimensional topological semantic graph. It measures whether the relative layout of components in the rendered graph matches the connectivity in the topological graph (e.g., through a differentiable graph matching algorithm) to ensure the correctness of the rendered logical structure. Specifically, the structural consistency loss is calculated as follows: First, the center points or key points of components are extracted from the rendered multi-channel feature map (or its semantic segmentation results) to construct a "rendered graph"; then, the two-dimensional topological semantic graph is used as the "target graph". The structural consistency loss L_struct is defined as a differentiable graph matching loss between two graphs. For example, it can use the cosine similarity loss of node embedding vectors based on graph neural networks, or a relaxed form of graph edit distance calculated based on optimal transmission (such as the Sinkhorn algorithm), so that the node layout and connectivity in the rendered graph are guided to match the target topology during the optimization process. Figure 1 To.

[0069] The cross-modal semantic and structural consistency loss also includes: physical property consistency loss, which is used to quantify the pixel-level differences between the physical data channels (e.g., thermal infrared and near-infrared channels) in the multi-channel feature map of the differentiable rendering projection and the corresponding physical data channels in the original multispectral image aligned with the semantic representation.

[0070] Specifically, the loss function is expanded into three parts. While calculating semantic consistency loss and structural consistency loss, the system also calculates physical property consistency loss, which is directly applied between the rendered multi-channel feature map and the original multispectral image serving as the ground truth. For example, the system extracts the thermal infrared channel of the rendered map (the value is obtained from the 3D geometric spectral field based on spatial coordinate queries) and the original thermal infrared image (already aligned through semantic representation), calculating the L2 norm (Euclidean distance) between them to penalize deviations between the rendered temperature and the true temperature. Similarly, this is done for the near-infrared channel to ensure that the rendered material reflectance is consistent with the true value.

[0071] By incorporating key physical information such as thermal infrared and near-infrared into the optimization target of registration, the alignment of the three-dimensional geometric spectral field no longer depends solely on geometric contours and semantic labels, but also on real physical readings, thus achieving high-precision registration at the physical level.

[0072] The system uses this total loss as the global optimization objective and iteratively and finely adjusts the internal network weights of the three-dimensional geometric spectral field and the six-degree-of-freedom pose parameters of the virtual camera through the backpropagation algorithm until the loss function converges to the minimum value.

[0073] Once optimization is complete, the 3D model and 2D image are aligned. The system can directly query and extract the 3D center points and feature point clouds of all geometric regions defined as connection points (such as screw and cable crimp points) from the registered 3D geometric spectral field, obtaining their precise spatial coordinates in the real-world coordinate system, which facilitates subsequent recognition tasks. The specific process of querying and extracting from the registered 3D geometric spectral field is as follows: First, a high-resolution 3D voxel grid is defined within the bounding box of the junction box; then, the geometric density value and semantic category of each voxel center point in the grid are queried in batches (assuming the model also learned the semantic output during training); then, all voxels with a geometric density greater than a preset threshold and whose semantic category is identified as "connection point" are selected; finally, the arithmetic mean of the 3D coordinates of all selected voxels is calculated to obtain the 3D center point of the connection point region, which serves as its spatial coordinates.

[0074] Furthermore, after accurately locating the three-dimensional coordinates of all key components, the system's multi-level diagnostic module begins to perform defect identification tasks. The inspection robot, based on a preset scanning strategy or preliminary diagnostic results, controls a multispectral camera to acquire a series of two-dimensional images from different perspectives and under different operating conditions (e.g., power-on and power-off states). For this image sequence, the system first applies a photometric correction algorithm (e.g., based on image histogram matching) to eliminate inconsistencies in pixel brightness and chromaticity caused by changes in illumination angle, intensity, and color temperature, ensuring that the image sequence has standardized illumination conditions. Next, the system uses the three-dimensional spatial coordinates of the connection points obtained in the previous step as three-dimensional anchor points, and projects these three-dimensional spatial regions back onto each photometrically corrected two-dimensional image through a camera projection matrix, automatically extracting multi-view image blocks corresponding to the same physical wiring area. Subsequently, based on the camera pose parameters corresponding to each image block, the system uses a projection transformation model to calculate the projection transformation matrix required to transform each view image block to a unified, perspective-distortion-free two-dimensional orthophoto plane, and applies this matrix to perform geometric correction and sub-pixel-level alignment. During the fusion phase, a weighted average based on signal-to-noise ratio evaluation and a pixel-level optimization strategy based on image gradient were adopted for the overlapping areas of the aligned image blocks. This strategy can suppress sensor noise and atmospheric disturbance blurring, and also enhance the details of real defects that appear stably (such as tiny cracks or corrosion spots). Finally, an enhanced wiring information map with spatiotemporal consistency in texture, spectral color and geometry is constructed.

[0075] The spatiotemporal consistency-enhanced infographic provides stable and reliable texture, color, and structural information, offering accurate data archives for longitudinal comparison and trend analysis of device status. It elevates diagnosis from a single snapshot to continuous and predictable health status monitoring, and storing monitoring results in the data archive helps improve the accuracy of identification.

[0076] In some embodiments of the present invention, in step S4, the step of using a feature encoding network trained by self-supervised contrastive learning based on normal samples to perform multi-scale feature extraction and anomaly assessment on the wiring information diagram, and generating a wiring defect saliency map and wiring status diagnosis results includes: In the pre-training phase of the implicit neural representation model, normal wiring information diagrams without wiring defects are used as training samples. For each normal wiring information diagram, a data augmentation operation is performed. The data augmentation operation includes at least one or more of random cropping, random flipping, color perturbation, and blurring. Positive sample items are constructed based on sample pairs formed by the same original wiring information diagram in the augmented image, and negative sample items are constructed based on sample pairs formed by different original wiring information diagrams. The contrastive learning loss is calculated based on the feature distance between the positive sample items and the negative sample items, and the feature encoding network parameters are updated. During the diagnostic phase, the spatiotemporal consistency enhanced wiring information map is input into the feature encoding network to obtain multi-scale feature representations. Based on the similarity or distance between the multi-scale features and the normal sample feature set obtained in the pre-training phase, the anomaly score of each spatial location is calculated to construct a wiring defect saliency map. According to the comparison results of the anomaly score of each wiring region in the wiring defect saliency map with the preset threshold, the wiring regions are classified to obtain the wiring status diagnostic results.

[0077] In the above implementation, a self-supervised contrastive learning defect feature model is employed to perform cross-scale automated defect identification on this information-rich enhanced wiring infographic. The greatest advantage of this model lies in its unique training paradigm; it does not require any samples with defect annotations, and the training process uses only a massive amount of normal wiring infographics collected from healthy devices without any defects. A series of complex data augmentation transformations are applied to each normal training sample, including random cropping, random horizontal flipping, random color jittering, and Gaussian blurring. The contrastive learning loss function specifically adopts the InfoNCE loss function, which aims to maximize the cosine similarity of the feature vectors of positive sample pairs (two different enhanced versions from the same image) while minimizing their cosine similarity with the feature vectors of all negative sample pairs within the batch. Similarity; the difference metric is achieved by calculating the average Euclidean distance between each feature vector in the feature map to be diagnosed and its K nearest neighbor features in the normal feature space learned during the pre-training phase, where the average Euclidean distance is the anomaly score; the system generates multiple positive sample pairs with similar content but different appearances for each sample, while samples from different original images constitute negative sample pairs; based on the contrastive learning loss function, the model is trained to bring the positive sample pairs closer in the latent feature space while pushing away the negative sample pairs; after such large-scale self-supervised pre-training, the model's encoder learns how to highly and tightly map the ever-changing but essentially normal wiring diagram features onto a specific, compact manifold in the latent space, which constitutes the feature distribution of the normal pattern in the latent space.

[0078] Furthermore, during actual diagnosis, the enhanced wiring information map to be detected is input into a multi-scale feature extraction network with a built-in feature pyramid structure to simultaneously acquire a series of feature maps representing different receptive fields and spatial resolutions, thereby enabling simultaneous attention to large-area color difference anomalies and pixel-level micro-cracks. The system performs pixel-wise and region-wise difference measurements between the feature map at each level and the normal feature space learned during the pre-training phase. In this embodiment, the normal feature space is constructed as a feature repository during the pre-training phase, which stores multi-scale feature vectors from all normal training samples. During recognition, the difference measurement is defined as the average distance (e.g., L2 distance) between the feature map to be detected (e.g., the feature vector f_ij at position i,j) and its K nearest neighbor (K-NN) normal features in the feature repository. If the distance exceeds a preset threshold, the location f_ij is judged as an anomaly, and an anomaly score is calculated based on the distance. Any feature whose mapping position in the latent space deviates from the compact normal manifold will be judged as an anomaly and receive a higher anomaly score. Subsequently, the anomaly score maps at all scales are upsampled to the same original resolution as the input information map, and weighted and summed according to the sensitivity of each scale to different types of defects to generate a pixel-level defect saliency map. In the defect saliency map, all potential defect regions will be clearly highlighted in the form of a heatmap. The system only needs to set a dynamic threshold based on statistics or experience to accurately locate and segment all defect regions of different sizes and types, and complete the final identification and diagnosis.

[0079] The system incorporates a feature pyramid structure and uses weighted fusion of multi-scale anomaly scores to significantly improve the accuracy and robustness of defect detection. It can also generate intuitive pixel-level defect location maps, achieving efficient, accurate, and automated defect diagnosis without manual annotation.

[0080] In the application scenario of this embodiment, firstly, the data acquisition module and the semantic representation module collect and deeply analyze multi-source, multi-modal field data to construct a semantic representation of the wiring area that is unaffected by environmental interference and accurate to the pixel level. Next, the 3D registration module constructs a 3D geometric spectral field containing physical laws based on this semantic representation and multispectral fusion point cloud, and achieves accurate registration of the 2D image and 3D model through differentiable rendering technology to obtain the precise spatial coordinates of all key connection points. Then, the multi-level diagnosis module uses these spatial coordinates to fuse multi-view image sequences to generate an ultra-high resolution enhanced wiring information map, and combines time-series comparison and self-supervised learning models to achieve accurate identification of static and dynamic evolution defects. Finally, based on the identification results, the system performs root cause diagnosis of faults, life prediction, and automatically generates intelligent maintenance work orders through the built-in expert knowledge graph. The above process, from front-end multimodal perception and environmental understanding, to mid-end high-precision 3D digital twin construction, and then to back-end deep diagnosis and predictive maintenance decision-making, constitutes a complete, closed-loop, cognitively intelligent joint junction box wiring status identification and diagnosis system.

[0081] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention achieves junction box status perception by fusing two-dimensional images and multispectral three-dimensional point cloud data. The solution utilizes cross-domain feature decoupling and a dual attention mechanism to significantly improve the accuracy of image semantic understanding. Furthermore, by introducing differentiable rendering technology for three-dimensional registration, this solution can deeply couple and reverse-optimize two-dimensional semantic information with three-dimensional geometric spectral information, thereby obtaining high-precision spatial localization of connection points and solving the problem of inaccurate cross-modal data registration in traditional methods. Finally, by combining spatiotemporal consistency enhancement technology and a self-supervised contrastive learning model, defect diagnosis becomes not only comprehensive and accurate but also adaptable to changes at different scales and in complex environments, significantly improving the automation level, reliability, and accuracy of wiring status identification.

[0082] Figure 2 This is a flowchart illustrating a combined junction box wiring status identification and diagnosis system according to an embodiment of the present invention.

[0083] Example 2, as Figure 2 As shown, the present invention also provides a combined junction box wiring status identification and diagnosis system, including: a data acquisition module S11, a semantic representation module S12, a three-dimensional registration module S13, and a multi-level diagnosis module S14.

[0084] The data acquisition module S11 is used to acquire multi-view two-dimensional image data of the junction box and the surrounding area of ​​the junction box; and to acquire multispectral fused point cloud data spatially registered with the multi-view two-dimensional image data. The semantic representation module S12, connected to the data acquisition module, is used to perform feature encoding on the two-dimensional image. Based on the cross-domain feature decoupling mechanism, the feature encoding is decoupled into domain-invariant semantic feature components and domain-related style feature components in the latent space. Based on adversarial constraints and orthogonal constraints, the domain-related information in the semantic features is suppressed. In the hierarchical feature extraction network, spatial attention and channel attention are applied to the intermediate feature maps of each level obtained from the domain-invariant semantic feature components to generate the semantic segmentation results of each wiring component of the joint junction box and a two-dimensional topological semantic map. The three-dimensional registration module S13, connected to the semantic representation module and the data acquisition module, is used to input the semantic segmentation result and the multispectral fusion point cloud into a pre-trained implicit neural representation model, and to establish a three-dimensional geometric spectral field of the joint junction box in the implicit neural representation model; under the preset virtual camera pose, a rendering feature map is generated based on the three-dimensional geometric spectral field through differentiable rendering; and the three-dimensional spatial coordinates of each wiring component and connection point are obtained according to the semantic and structural correspondence between the rendering feature map, the two-dimensional image and the two-dimensional topological semantic map. The multi-level diagnostic module S14 is connected to the three-dimensional registration module and the data acquisition module. It is used to extract and register wiring area image blocks from multi-time and multi-view images with the three-dimensional spatial coordinates as anchor points, construct a spatiotemporal consistency enhanced wiring information map, and use a feature encoding network trained by self-supervised contrastive learning based on normal samples to perform multi-scale feature extraction and anomaly evaluation on the wiring information map, and generate a wiring defect saliency map and wiring status diagnostic results.

[0085] The combined junction box wiring status identification and diagnosis system provided by this invention accurately identifies such unknown aging defects through deep learning and multimodal sensing fusion technology, achieving true predictive maintenance.

[0086] This invention is based on four modules: data acquisition, semantic representation, 3D registration, and multi-level diagnosis. The data acquisition module collects two-dimensional images and three-dimensional spatial feature data of the junction box. The semantic representation module generates semantic representations using cross-domain feature decoupling and dual attention mechanisms. Next, the 3D registration module performs high-precision pattern matching between the semantic objects identified in the images and the 3D spatial feature data, and confirms the spatial position of the identified components by optimizing the cross-modal consistency target. Finally, the multi-level diagnosis module integrates the recognition results and spatial position information of multi-view sequence images to construct spatiotemporal consistency features for state discrimination, and uses a self-supervised contrastive learning model to perform cross-scale defect pattern recognition to obtain diagnostic results.

[0087] The implementation process of the combined junction box wiring status identification and diagnosis system provided by the present invention begins with the high-fidelity acquisition of field data.

[0088] The primary task of the semantic representation module of the system is to overcome the severe domain shift problem caused by environmental factors such as rust, stains, and uneven lighting. Traditional supervised learning-based image segmentation algorithms suffer a sharp decline in performance, or even complete failure, when faced with on-site contaminated images that differ greatly in appearance from the training samples. The semantic representation module of this invention fundamentally solves this problem through a cross-domain feature decoupling mechanism. When an image of a terminal block covered with rust and salt spray crystals is input into a deep convolutional neural network encoder, this decoupling mechanism is activated. This mechanism has learned a large amount of data from clean factory samples and on-site contaminated samples during the pre-training stage. Through a designed adversarial loss function and orthogonality constraints, it forces the network to clearly decompose the mixed features extracted from the image into two independent components. One is a domain-invariant semantic feature component, which carries the inherent and essential geometric and topological properties of objects such as terminals, cable insulation, and screws, and is unaffected by their surface conditions. The other is a domain-related style feature component, which captures all appearance information related to the environment, such as the texture of rust, the reflection of salt spray, or the color of oil stains. After decoupling, the system will selectively discard the domain-related style feature components that represent environmental interference, and only send the pure domain-invariant semantic feature components to the decoder part of the network for subsequent processing.

[0089] The intermediate feature maps generated by the decoder can outline the contours and boundaries of each component, removing the interference of smudged images and ensuring the accuracy of semantic understanding.

[0090] Meanwhile, to enhance the robustness and information density of semantic representation, the system introduces a dual attention weighting mechanism of spatial and channel attention in the hierarchical feature extraction network. This mechanism adaptively adjusts the feature attention according to task requirements. For example, in moisture detection-related tasks, the channel attention network increases the weight of the short-wave infrared channel feature map, making the semantic information related to moisture or water accumulation more significant. At the same time, the spatial attention network focuses on high-risk physical locations determined based on prior knowledge, such as the interface between terminals and cables, the contact area between screws and pressure plates, and the edge of the sealing ring.

[0091] The feature map is weighted through a dual attention mechanism to generate a semantic representation map with high information density and no environmental interference. Based on this two-dimensional semantic foundation, the three-dimensional registration module is launched. Its task is to spatially align the two-dimensional semantics with the three-dimensional multispectral physical information. This module adopts an implicit neural representation model, which constructs a differentiable three-dimensional geometric spectral field, learns the mapping from three-dimensional spatial coordinates to geometric entity density, and masters the correspondence between multi-dimensional spectral vectors and their respective semantic categories.

[0092] Through the calculations of this model, the system ultimately constructs a registered and queryable three-dimensional spectral model. In this model, the semantic, geometric, and physicochemical properties of any point in space are unified.

[0093] The system's multi-level diagnostic module performs high-precision self-supervised defect diagnosis aimed at identifying unknown aging defects. To ensure the diagnostic model acquires optimal input information, the system first employs multi-view sequence fusion technology. This involves fusing two-dimensional image sequences from different shooting angles, with dynamic shadows and geometric distortions, through photometric correction and projection transformation, into a high-resolution, shadow-free, and distortion-free standardized two-dimensional orthophoto view—a spatiotemporal consistency-enhanced wiring information map. This processing suppresses random noise and surface reflections and enhances the visibility of fine structural details such as microcracks. The core of the diagnosis utilizes a self-supervised contrastive learning defect feature model. Its pre-training phase does not rely on labeled defect samples but instead learns from a large number of defect-free, normal images from new products and healthy, in-service equipment to understand the potential health wiring status. The system constructs a baseline feature space for normal patterns by using a compact distribution in the feature space. During diagnosis, the enhanced wiring information map to be detected is fed into a multi-scale feature extraction network. The system calculates the difference between the feature vector and the predefined normal feature space pixel by pixel and region by region at each scale. Any feature whose mapping position in the latent space deviates from the compact manifold is judged as abnormal and receives a high abnormality score. Finally, the system integrates the abnormality score maps at all scales to generate a global saliency map that indicates the location and intensity of defects. This method improves the system's generalization ability by constructing a normal feature space and detecting deviations, rather than relying on the identification of specific defect types.

[0094] This anomaly detection-based strategy does not rely on a massive, labeled defect sample library, enabling it to efficiently and accurately identify previously unseen, atypical, or early-stage minor defects. This improves the scope and sensitivity of defect detection and ensures reliable identification of both known and unknown fault modes.

[0095] Specifically, in this embodiment, a region on the spectrum is highlighted, corresponding to a slightly darker terminal surface in the original image. After the system automatically locates the abnormal region through threshold segmentation, it immediately activates the anomaly attribution mechanism. It calls the data from the three-dimensional registration module to query the original physical properties of the abnormal region in the three-dimensional geometric spectral field. The query results show that the short-wave infrared spectral value of the region is abnormally high, far exceeding the normal threshold. This is strong physical evidence of water accumulation. Based on this, the system generates a clear diagnostic report.

[0096] In summary, this embodiment fully demonstrates that the system of the present invention overcomes environmental interference through cross-domain semantic representation technology in extremely harsh industrial environments, achieving accurate understanding of equipment; and by utilizing a self-supervised learning model, it breaks away from dependence on a predefined defect library, successfully identifying unknown aging defects in their nascent stage caused by environmental factors. This enables higher-level and higher-dimensional predictive maintenance diagnosis, providing strong technical support for ensuring the safe and stable operation of critical infrastructure.

[0097] Example 3: The present invention also provides a combined junction box wiring status identification and diagnosis device. The device includes a computer device, which includes a processor and a memory. The processor stores computer instructions. When the computer instructions are executed, the device implements the combined junction box wiring status identification and diagnosis method.

[0098] Example 4, as Figure 3 As shown, the present invention also provides an electronic device 100 for implementing a method for identifying and diagnosing the wiring status of a combined junction box.

[0099] The electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on at least one processor 102, and at least one communication bus 104.

[0100] The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the combined junction box wiring status identification and diagnosis method described in the first aspect of the present invention by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101.

[0101] The memory 101 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the electronic device 100 (such as audio data), etc. In addition, the memory 101 may include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0102] At least one processor 102 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 102 may be a microprocessor or any conventional processor. Processor 102 is the control center of electronic device 100, connecting various parts of electronic device 100 via various interfaces and lines.

[0103] The memory 101 in the electronic device 100 stores multiple instructions to implement a combined junction box wiring status identification and diagnosis system, and the processor 102 can execute multiple instructions to achieve the following: Acquire multi-view two-dimensional image data of the junction box and its surrounding area; acquire multispectral fused point cloud data spatially registered with the multi-view two-dimensional image data; The two-dimensional image is feature-encoded. Based on the cross-domain feature decoupling mechanism, the feature encoding is decoupled into domain-invariant semantic feature components and domain-related style feature components in the latent space. Based on adversarial constraints and orthogonal constraints, the domain-related information in the semantic features is suppressed. In the hierarchical feature extraction network, spatial attention and channel attention are applied to the intermediate feature maps of each level obtained from the domain-invariant semantic feature components to generate the semantic segmentation results of each wiring component of the joint junction box and a two-dimensional topological semantic map. The semantic segmentation results and multispectral fusion point cloud are input into a pre-trained implicit neural representation model, and a three-dimensional geometric spectral field of the joint junction box is established in the implicit neural representation model. Under the preset virtual camera pose, a rendering feature map is generated based on the three-dimensional geometric spectral field through differentiable rendering. According to the semantic and structural correspondence between the rendering feature map, the two-dimensional image and the two-dimensional topological semantic map, the three-dimensional spatial coordinates of each wiring component and connection point are obtained. Using the three-dimensional spatial coordinates as anchor points, wiring area image blocks are extracted and registered from multi-time and multi-view images to construct a spatiotemporal consistency enhanced wiring information map. Using a feature encoding network trained by self-supervised contrastive learning based on normal samples, multi-scale feature extraction and anomaly assessment are performed on the wiring information map to generate a wiring defect saliency map and wiring status diagnosis results.

[0104] Example 5: If the modules / units integrated in the electronic device 100 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, and read-only memory (ROM).

[0105] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0106] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0107] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0108] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0109] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A jointing box jointing state recognition diagnosis method, characterized by, The method comprises: Obtaining multi-view two-dimensional image data of the joint terminal box and the peripheral area of the joint terminal box, and obtaining multi-spectral fused point cloud data that is spatially registered with the multi-view two-dimensional image data; Feature coding is performed on the two-dimensional image, the feature coding is decoupled into domain-invariant semantic feature components and domain-related style feature components in a latent space based on a cross-domain feature decoupling mechanism, domain-related information in the semantic features is inhibited based on an adversarial constraint and an orthogonal constraint, spatial attention and channel attention are applied to intermediate feature maps at each level obtained from the domain-invariant semantic feature components in a hierarchical feature extraction network, and a semantic segmentation result of each terminal component of the joint terminal box and a two-dimensional topological semantic graph are generated; The semantic segmentation result and the multi-spectral fused point cloud are input into a pre-trained implicit neural representation model, a three-dimensional geometric spectral field of the joint terminal box is established in the implicit neural representation model, and a rendering feature map is generated by differentiable rendering based on the three-dimensional geometric spectral field under a preset virtual camera pose; and three-dimensional spatial coordinates of each terminal component and a connection are obtained according to a semantic correspondence and a structural correspondence between the rendering feature map, the two-dimensional image and the two-dimensional topological semantic graph. Taking the three-dimensional spatial coordinates as anchor points, terminal region image blocks are extracted and registered from multi-time and multi-view images to construct a spatio-temporal consistency enhanced terminal information graph, a feature coding network trained based on normal samples through self-supervised contrastive learning is used to perform multi-scale feature extraction and abnormality evaluation on the terminal information graph, and a terminal defect saliency map and a terminal state diagnosis result are generated.

2. The jointing box connection state recognition diagnosis method according to claim 1, characterized by, It comprises: Three-dimensional point cloud data of the joint terminal box and the peripheral area thereof is collected by using a three-dimensional laser scanner, multi-spectral images covering visible light and infrared bands are synchronously collected by using a multi-spectral camera, and the multi-view multi-spectral images and the three-dimensional point cloud data are spatially registered and aligned in a unified coordinate system based on a joint calibration model established based on preset sensor internal parameters and external parameters to generate the multi-spectral fused point cloud.

3. The jointing box connection state recognition diagnosis method according to claim 1, characterized by, The process of decoupling the feature coding into the domain-invariant semantic feature components and the domain-related style feature components based on the cross-domain feature decoupling mechanism comprises: A feature extraction network including an encoder, a semantic feature branch and a style feature branch is constructed, and a domain classifier is connected at an output end of the semantic feature branch; The domain classifier outputs a classification result and calculates a classification loss according to an environmental domain label; In the training process of the encoder, the encoder is updated in parameters based on an adversarial training strategy, a parameter update direction of the encoder is opposite to a parameter update direction of reducing the classification loss, and features output by the encoder are divided into the domain-invariant semantic feature components and the domain-related style feature components.

4. The jointing box connection state recognition diagnosis method according to claim 3, characterized by, When the encoder is trained, the following steps are further included: An inter-information estimate value is calculated between the semantic feature components and the style feature components, and an inter-information loss is constructed to punish information redundancy therebetween. And a reconstruction loss is constructed based on a reconstruction error between an image reconstructed by the decoder and the original two-dimensional image, and the encoder and the decoder are optimized jointly based on the mutual information loss and the reconstruction loss, to obtain an encoding result in which a domain-related information component in semantic features is weakened.

5. The joint closure connection state identification diagnosis method according to claim 1, characterized by, In the hierarchical feature extraction network, spatial attention and channel attention are applied to the intermediate feature maps in each level obtained by the domain-invariant semantic feature component, and the semantic segmentation result and the two-dimensional topological semantic graph of each wiring assembly of the joint wiring box are generated by combining the spatial attention and the channel attention, including: The channel attention network is applied to the intermediate feature maps output by each layer of the decoder in the hierarchical feature extraction network, and the channel attention network obtains a channel weight vector through a global pooling operation and weights each channel; A two-dimensional attention mask is generated in the spatial dimension of the feature map by the spatial attention network, and each spatial position of the intermediate feature map is weighted position by position to obtain an attention-weighted feature map, in which the feature response value of the wiring assembly region is higher than that of the background region.

6. The joint closure terminal status identification diagnostic system of claim 1, wherein, In the implicit neural representation model, the three-dimensional geometric spectral field of the joint wiring box is established, including: An implicit neural representation model with a multilayer perceptron as the main structure is constructed, three-dimensional spatial coordinates after position encoding processing and feature vectors corresponding to the virtual observation viewing angle direction are taken as inputs, and the geometric density and multispectral feature values of the corresponding spatial points are output, and the rendering feature map is generated based on the geometric density and multispectral feature values through a differentiable rendering.

7. The joint closure terminal status identification diagnostic system of claim 6, wherein, The training method of the pre-trained implicit neural representation model includes: The gradient norm of the geometric density field in the three-dimensional geometric spectral field within the neighborhood of the surface of the joint wiring box is calculated, and a surface normal gradient constraint loss is constructed based on the gradient norm; For three-dimensional spatial points belonging to the same semantic category, the difference between the corresponding predicted multispectral feature values is calculated, and a semantic spectral internal consistency loss is constructed; In the parameter updating process, the weighted sum of the reconstruction loss, the surface normal gradient constraint loss and the semantic spectral internal consistency loss is taken as the total loss, and the parameters of the implicit neural representation model are iteratively updated.

8. The joint closure terminal status identification diagnostic system of claim 6, wherein, The training method of the pre-trained implicit neural representation model further includes: The difference between the physical data channel in the multi-channel feature map obtained by differentiable rendering of the three-dimensional geometric spectral field and the corresponding physical data channel in the original multispectral image after calibration alignment is calculated at the pixel level, and a physical property consistency loss is constructed based on the difference; In the parameter updating process, the weighted sum of the physical property consistency loss, the semantic consistency loss and the structural consistency loss is taken as the optimization objective, and the parameters of the implicit neural representation model are iteratively updated.

9. The joint closure connection state identification diagnosis method according to claim 1, characterized by, The feature encoding network trained based on normal samples through self-supervised contrastive learning is used to perform multi-scale feature extraction and abnormality evaluation on the wiring information graph, to generate a wiring defect saliency map and a wiring state diagnosis result, including: In the pre-training phase of the implicit neural representation model, a normal wiring information graph without wiring defects is used as a training sample, Data augmentation operations are performed on each normal wiring information map, the data augmentation operations at least including one or more of random cropping, random flipping, color disturbance and blurring, positive sample items are constructed based on sample pairs formed by the same original wiring information map in the augmented images, negative sample items are constructed by sample pairs formed by different original wiring information maps, a contrast learning loss is calculated according to the feature distance of the positive sample items and the negative sample items, and the feature encoding network parameters are updated; In the diagnosis stage, the spatiotemporal consistency enhanced wiring information map is input into the feature encoding network to obtain a multi-scale feature representation, an abnormality score of each spatial position is calculated based on the similarity or distance between the multi-scale feature and the normal sample feature set obtained in the pre-training stage, a wiring defect saliency map is constructed, and according to the comparison result of the abnormality score of each wiring area in the wiring defect saliency map and the preset threshold, the wiring area is classified to obtain a wiring state diagnosis result.

10. A combined junction box wiring status identification and diagnostic device, characterized in that, The device includes a computer device including a processor and a memory, and the processor has stored therein computer instructions which, when executed, implement the joint joint box wiring state identification and diagnosis method according to any one of claims 1 to 9.

Images