A visual and physical perception fusion fault detection system and method for off-board bolts

Abstract

The application discloses a kind of visual and physical perception fusion fault detection systems and methods of undercarriage bolt, belong to rail transit intelligent inspection and computer vision cross technical field.System includes image acquisition module, visual detection module, physical sensing node, space mapping module, time synchronization module, fusion determination module and result output module.After system obtains undercarriage image, using the main network containing embedded DWC-RepConv, TEAM-P1 fusion module and the improved RT-DETR network of enhanced detection head extract small target fine-grained features, output position detection frame and visual confidence;According to detection frame in space mapping table, match physical sensing node;Based on time stamp, extract synchronous physical features;Finally, the fusion confidence is calculated in combination with visual prediction result and physical feature, and the determination result is output.The application solves the problem that bolt is difficult to locate under the complex environment of undercarriage, the high false detection and missed detection rate and the low reliability of single perception.

Description

Technical Field

[0001] This invention relates to the field of rail transit safety inspection technology, and in particular to a fault detection system and method for under-vehicle bolts that combines visual and physical perception. Background Technology

[0002] With the rapid development of modern rail transit systems, the requirements for the safety and stability of train operation are increasing. Bolts are extensively used as basic connecting components in critical parts such as the train undercarriage and bogies. Due to the harsh working environment of trains, including high-speed operation, severe vibration, weathering, and temperature fluctuations, these undercarriage bolts are highly susceptible to mechanical failures such as loosening, corrosion, breakage, damage, or detachment. If these failures are not detected and addressed promptly, they can easily trigger a chain reaction. Currently, for routine inspections of the condition of train undercarriage bolts, some scenarios are beginning to introduce target detection algorithms to automate the detection of images captured by onboard inspection equipment.

[0003] However, existing computer vision-based bolt fault detection technologies still face significant shortcomings in real-world applications in complex vehicle undercarriage environments: First, visual features degrade severely in extreme environments, such as poor lighting conditions under the vehicle and oil stains often adhering to equipment surfaces, leading to high false positive and false negative rates; second, small targets are often stuck to the background, making localization difficult, and existing detection algorithms easily lose fine-grained features after multiple downsampling operations; third, the expression of micro-features of faults is insufficient, as early faults often manifest as weak features such as slight loosening or localized corrosion, which are not adequately addressed by the backbone network of existing detection models; finally, the single visual perception modality has a reliability bottleneck, and visual algorithms may misjudge when the bolt is internally fractured but its appearance remains unchanged, or when it is obstructed by the viewing angle. Summary of the Invention

[0004] The purpose of this invention is to provide a fault detection system and method for under-vehicle bolts that integrates visual and physical perception, in order to solve the technical problems of difficulty in extracting fine-grained fault features of bolt targets in the under-vehicle environment, and the low reliability of detection caused by the single visual detection modality in the presence of oil stains and complex backgrounds.

[0005] In a first aspect, the present invention provides a fault detection system for under-vehicle bolts that combines visual and physical perception, comprising: The image acquisition module is used to acquire images of the underside of the vehicle; Physical sensing nodes are used to collect physical state signals of the physical area under the vehicle; The visual inspection module is used to process the image of the bottom of the vehicle and output the location detection box of the bolt target, the visual fault category and the visual confidence level; The spatial mapping module is used to find the corresponding physical sensor node number in a preset spatial mapping table based on the position detection box. The time synchronization module is used to extract the physical state signals collected by the physical sensing nodes within the same time window based on the timestamp of the vehicle bottom image, and obtain the physical feature vector. The fusion determination module is used to calculate the fusion confidence of each fault category based on the visual fault category, the visual confidence, and the physical feature vector, and determine the final bolt fault category. The results output module is used to output the detection results, which include the final bolt fault category.

[0006] As an optional implementation of the first aspect of this application, the visual detection module incorporates an improved RT-DETR network, which includes: a backbone network for extracting local texture and shallow spatial features from the vehicle bottom image and outputting basic features, wherein the backbone network is a Swing Transformer network with an embedded DWC-RepConv composite convolution enhancement module; a TEAM-P1 feature fusion module for receiving the basic features and performing intra-scale interaction and cross-scale fusion to output enhanced features; and a multi-scale perception enhancement detection head for making predictions based on the enhanced features and outputting the bolt target's location detection box, visual fault category, and visual confidence level.

[0007] As an optional implementation of the first aspect of this application, the backbone network includes four hierarchical feature extraction stages. In the improved Swing Transformer Block of each feature extraction stage, the DWC-RepConv composite convolution enhancement module is embedded in the local window attention calculation output and before the MLP feedforward layer. During the model training stage, the DWC-RepConv composite convolution enhancement module enhances the input features through parallel depthwise separable convolution branches, reparameterized convolution branches, and identity mapping branches. During the model inference stage, the branches are equivalently fused into a single convolution for feature output.

[0008] As an optional implementation of the first aspect of this application, the TEAM-P1 feature fusion module includes: a P1 high-resolution feature reconstruction unit, used to perform channel expansion and sub-pixel rearrangement on the shallow features in the basic features to restore local high-resolution features; an axially sensitive OFP fusion unit, used to model and fuse the relationship between bolts in the same row and the structural relationship between bolts and the frame using horizontal and vertical paths respectively, and output axially sensitive fusion features; and a TEAM dynamic texture enhancement attention unit, used to process the axially sensitive fusion features through dilated convolution, deformable convolution and multi-scale texture branching and fuse them based on dynamic weights, and output the enhanced features.

[0009] As an optional implementation of the first aspect of this application, the multi-scale perception enhancement detection head includes: a RoI perception enhancement unit, configured to generate a reference candidate box based on the target query vector output by the decoder, extract local region features from the enhancement features based on the reference candidate box and fuse them with the original target query vector, and output an enhanced target query vector; a localization branch, configured to predict a basic bounding box based on the enhanced target query vector, and predict a learnable bounding box offset to adaptively correct the basic bounding box, and output the location detection box; and a classification branch, configured to calculate the classification channel attention weights using a residual attention mechanism and enhance the target query vector, and output the visual fault category and visual confidence.

[0010] As an optional embodiment of the first aspect of this application, the physical sensing node includes at least one of a vibration sensor, a temperature sensor, and a conduction state detection circuit; the physical feature vector includes at least one of vibration anomaly features, temperature anomaly features, conduction state, and resistance or impedance anomaly features extracted by filtering the physical state signal.

[0011] As an optional implementation of the first aspect of this application, the specific determination process of the fusion determination module is as follows: based on the physical feature vector, calculate the vibration anomaly score, temperature anomaly score, and continuity / impedance anomaly score respectively; for each fault category, weight the visual fault category prediction probability of the corresponding category and the anomaly score by using preset fusion parameters to obtain the fusion confidence of the fault category; and take the fault category corresponding to the maximum fusion confidence as the final bolt fault category.

[0012] Secondly, this invention provides a method for detecting faults in under-vehicle bolts by fusing visual and physical perception, comprising the following steps: The image acquisition module acquires images of the bottom of the vehicle, and the physical state signals of the physical area under the vehicle are collected by the physical sensor nodes. The vehicle underside image is processed by a visual inspection module, which outputs the bolt target location detection box, visual fault category, and visual confidence level. The spatial mapping module uses the position detection box to find the corresponding physical sensor node number in the preset spatial mapping table. The physical feature vector is obtained by extracting the physical state signals collected by the physical sensor nodes within the same time window based on the acquisition timestamp of the vehicle bottom image by the time synchronization module. The fusion determination module calculates the fusion confidence of each fault category based on the visual fault category, the visual confidence, and the physical feature vector, and determines the final bolt fault category. The results output module outputs the detection results, which include the final bolt fault category.

[0013] As an optional implementation of the second aspect of this application, before searching for the physical sensor node number, the method further includes: establishing the spatial mapping table during the system initialization phase, wherein the spatial mapping table includes the bolt detection area number, the corresponding physical sensor node number, and the mapping relationship of the position range of the area in the vehicle under coordinate system.

[0014] As an optional implementation of the second aspect of this application, the determination step of the fusion determination module further includes: determining whether the visual fault category is consistent with the abnormal score of the physical feature vector; if a state occurs where the visual prediction is normal but the abnormal score exceeds the threshold, or the visual prediction is faulty but the abnormal score is below the threshold, then a visual-physical conflict status identifier is output and a manual review prompt is triggered.

[0015] Thirdly, embodiments of this application provide an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the method described in the second aspect.

[0016] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the second aspect.

[0017] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention proposes a spatiotemporal mapping fusion judgment mechanism for visual detection results and physical sensing nodes. It performs spatial alignment by acquiring a spatial mapping table, extracts physical signals based on timestamps for time synchronization, and makes a comprehensive judgment on the failure of bolts under the vehicle through a method of "visual judgment, physical verification, and conflict review". It solves the reliability bottleneck under a single visual modality and effectively reduces missed detections and false detections under conditions of uneven lighting and oil stains.

[0018] (2) This invention introduces a Swin Transformer backbone network embedded with DWC-RepConv into the visual detection network. It takes into account the global context modeling capability of the Transformer structure and the local texture enhancement capability of DWC-RepConv. It can not only accurately extract fine-grained features such as bolt edges, threads, and fracture notches, but also has lightweight inference capability, which is conducive to edge deployment.

[0019] (3) The present invention designs the TEAM-P1 feature fusion module, which restores the edge information in the shallow high-resolution features through super-resolution reconstruction, accurately models the spatial arrangement relationship of bolts in the same row by using axial sensitive fusion, and highlights the fault-sensitive areas such as rust and cracks by dynamic texture enhancement, which greatly improves the model’s sensitivity to early weak faults.

[0020] (4) The present invention constructs a multi-scale perception enhancement detection head, which integrates local features through the RoI perception mechanism and combines learnable bounding box correction offset to adaptively correct the initial prediction box, thus solving the problem of positioning offset error caused by the adhesion of small targets and background in small target detection. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall structure of the lightweight Swin Transformer backbone network in an embodiment of the present invention; Figure 2 This is a schematic diagram of the improved Swing Transformer Block in an embodiment of the present invention; Figure 3 This is a schematic diagram of the TEAM-P1 feature fusion module in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of the multi-scale sensing enhancement detection head in an embodiment of the present invention; Figure 5 This is a schematic diagram of the collaborative identification and fusion determination process in an embodiment of the present invention; Figure 6 This is a flowchart illustrating the overall process structure of the undercarriage bolt visual and physical perception fusion fault detection method in this embodiment of the invention. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0024] This invention provides a fault detection system and method for undercarriage bolts that integrates visual and physical perception. The system is designed for inspection scenarios involving undercarriage structures such as train undercarriages, bogies, connecting brackets, and equipment mounting seats. Addressing issues such as small bolt target size, complex background structures, uneven lighting, oil smudges, unclear early fault visual features, and insufficient reliability of single-vision detection, this invention proposes a detection scheme that integrates "visual detection and positioning—physical sensor node mapping—time window synchronization—multi-source feature fusion—fault determination output."

[0025] Example 1 The system of this invention includes an image acquisition module, a visual inspection module, physical sensor nodes, a spatial mapping module, a time synchronization module, a fusion judgment module, and a result output module. Specifically, the image acquisition module acquires images of the vehicle's underside; the visual inspection module outputs the location of the bolt target, the visual fault category, and the visual confidence level; the physical sensor nodes acquire physical signals such as vibration, temperature, conductivity, resistance, or impedance; the spatial mapping module establishes the correspondence between the visual inspection frame and the corresponding physical area under the vehicle and the sensor nodes; the time synchronization module extracts physical signals within the same time window based on the image acquisition timestamp; the fusion judgment module calculates the fusion confidence level for each fault category based on the visual inspection results and physical state characteristics; and the result output module outputs the final bolt fault category, fusion confidence level, alarm level, detection image, physical waveform, and review prompt information.

[0026] In the visual inspection module, this invention builds upon the RT-DETR inspection framework to further construct a lightweight visual inspection network adapted for detecting small targets such as bolts under vehicles. This network includes a lightweight SwinTransformer backbone network embedded with DWC-RepConv, a TEAM-P1 feature fusion module, and a multi-scale perception enhancement head to enhance the model's ability to identify fine-grained fault features such as bolt edges, threads, rust spots, fracture notches, missing boundaries, and loose displacement.

[0027] The technical implementation of this invention includes the following: I. Data Input and System Initialization Phase The system first acquires images of the vehicle's underside using an image acquisition module installed on a mobile inspection device, a vehicle under-inspection robot, or a fixed inspection platform. The image acquisition module may include an industrial camera, a high-definition camera, or other visual acquisition devices capable of acquiring images of the vehicle's underside structure. After preprocessing such as size normalization, brightness enhancement, noise suppression, and pixel standardization, the acquired vehicle underside images are input to the subsequent visual inspection module.

[0028] During the system initialization phase, a correspondence is established between the detection area of ​​the undercarriage bolts, the image coordinate range, the location of the physical structure under the vehicle, and the physical sensor node number, and a spatial mapping table is generated. This spatial mapping table is used in subsequent detection processes to associate the bolt detection frames output by the visual inspection module with the corresponding physical areas and physical sensor nodes under the vehicle, thus providing a basis for spatial alignment between the visual inspection results and the physical sensor signals.

[0029] A spatial mapping table can be represented as: in, This represents the detection area for the k-th bolt. This indicates the physical sensor node number corresponding to this area. This indicates the location range of the region in the vehicle under-structure coordinate system or the image coordinate system.

[0030] In a preferred embodiment, the spatial mapping table can be obtained through manual calibration, structural drawing calibration, or inspection equipment positioning calibration. For fixed detection areas, the system can pre-establish a one-to-one correspondence between bolt areas and sensor nodes; for mobile inspection scenarios, the system can combine camera viewpoint, inspection equipment pose, and vehicle undercarriage structure coordinates to convert the bolt detection positions in the image into physical areas under the vehicle, and then further match the corresponding sensor nodes.

[0031] Through the above initialization process, the system can accurately associate the visually detected bolt targets with the corresponding physical sensing nodes in the subsequent detection process, avoiding erroneous matching between visual targets and irrelevant physical signals, and providing a basis for subsequent visual-physical fusion judgment.

[0032] II. Visual inspection network adapted for detecting small targets such as bolts under vehicles The visual inspection module employs an improved RT-DETR network to detect bolt targets in vehicle underbody images, outputting the bolt target's location, visual fault category, and visual confidence level. Its purpose is to improve the localization accuracy and fault identification capability of small bolt targets in complex under-vehicle environments, providing reliable visual evidence for subsequent visual-physical fusion judgment. This improved RT-DETR network mainly consists of: a lightweight backbone network, a TEAM-P1 feature fusion module, and a multi-scale perception enhancement detection head.

[0033] 1. A Swing Transformer backbone network embedded with DWC-RepConv This section primarily addresses the challenges of "weak features of small targets, strong interference from complex backgrounds, and limited deployment of edge devices" in undercarriage bolt fault detection. Undercarriage bolts are typically small in size in images, surrounded by interference from oil stains, pipelines, shadows, metal reflections, and vehicle frame structures. While traditional CNN backbone networks can extract local textures, their global context modeling capabilities are insufficient. Although the standard Swin Transformer can capture contextual information through a window attention mechanism, it is computationally intensive and has limited ability to enhance fine-grained features such as bolt edges, threads, rust spots, and fracture notches.

[0034] Therefore, this invention embeds the DWC-RepConv composite convolution enhancement module into the Swing Transformer backbone network, enabling it to simultaneously possess global modeling, local texture enhancement, and lightweight deployment capabilities.

[0035] (1) Overall structure of the backbone network like Figure 1 As shown, the lightweight Swin Transformer backbone network used in this invention consists of a PatchPartition image slicing module, a Linear Embedding module, a four-stage improved SwinTransformer Block, and a Patch Merging downsampling module. The overall process is as follows: Input → Image Slicing Module → Linear Embedding Module → Stage 1 → Stage 2 → Stage 3 → Stage 4.

[0036] Let the input image be the undercarriage of the vehicle. Here, H and W represent the image height and width, respectively, and 3 represents the number of RGB image channels. First, the input image is divided into several non-overlapping segments. Image patch, each patch has the following dimensions after being flattened: The number of patches is Subsequently, each patch is mapped to a C-dimensional feature space using a linear embedding module to obtain the initial embedding features. The initial embedded features then sequentially enter four hierarchical feature extraction stages: Phase 1: For Local texture and shallow spatial features are extracted, and the spatial size is halved and the number of channels is expanded to 2C through a downsampling module, where C represents the feature dimension. The output is... .

[0037] Phase 2: Further extract mid-level structural features and output .

[0038] Stage 3: Used to extract higher-level semantic features and contextual relationships, outputting... .

[0039] Phase 4: In Based on this, advanced semantic modeling is performed to obtain the final backbone output. The output contains both spatial structural information and high-level semantic information of the bolt target, which serves as input for the subsequent TEAM-P1 feature fusion module.

[0040] (2) Improve the Swing Transformer Block like Figure 2 As shown, in each improved Swin Transformer Block, the input features are assumed to be: Where B is the batch size and c is the number of channels. First, LN normalization is performed on the input features: The normalized features are then divided into several local windows, and W-MSA or SW-MSA attention calculation is performed within each window. The present invention preferably sets the window size to... This allows attention calculations to focus more on the bolt itself and its adjacent area, reducing interference from the large background area under the vehicle. For features within the window, linear mapping is used to obtain: in, , , These represent the query vector, key vector, and value vector, respectively. , , These represent the query weight matrix, key weight matrix, and value weight matrix, respectively.

[0041] The formula for calculating attention is: in This represents the attention calculation function. Let represent the normalized exponential function, and d be the dimension of a single attention head. This is a learnable relative position bias matrix. The window attention output is then residually concatenated with the input features. in This represents the window attention residual output. This indicates multi-head self-attention calculation for the window. Adjacent blocks exchange information across windows via SW_MSA, in the following form: in This indicates a multi-head self-attention mechanism in the shift window.

[0042] By alternately stacking W_MSA and SW_MSA, the network can extract the local texture of the bolts and obtain the structural relationship between adjacent regions, thereby enhancing the model's feature representation ability in complex vehicle undercarriage environments.

[0043] (3) DWC-RepConv composite convolution enhancement To compensate for the insufficient representation of local texture details by the transformer structure, this invention embeds a DWC-RepConv composite convolution enhancement module after the attention module and before the MLP feedforward layer. Let the window attention residual output be... This module includes a depthwise separable convolution branch, a reparameterized convolution branch, and an identity mapping branch during the training phase.

[0044] Depthwise separable convolution branches (for low-cost extraction of local texture features): in This represents the output of the depthwise separable convolution branch. Represents depthwise convolution. This represents pointwise convolution.

[0045] Reparameterized convolution branches (used to enhance the representation of local spatial structure): in This represents the output of the reparameterized convolution branch. and They represent Convolution and convolution.

[0046] Identity mapping branch (used to preserve original features): in This represents the output of the identity mapping branch.

[0047] Therefore, the output of the DWC-RepConv module during the training phase is: Right now This structure enhances local features such as the bolt's hexagonal profile, thread texture, rust spots, fracture notches, and loose edge variations. During the inference phase, multi-branch structures can be merged into a single structure through reparameterization. convolution: in, This represents the equivalent convolution after reparameterization. This design allows the model to have stronger feature learning capabilities during training while maintaining low computational cost during deployment, thus improving deployment efficiency on mobile inspection devices or embedded edge platforms.

[0048] The enhanced feature F output by DWC-RepConv is passed through LN and MLP feedforward networks, and then the current output is obtained through residual connections: The output is then fed into subsequent Swing Transformer Blocks or Patch Merging modules to complete deeper feature extraction. Through this design, the backbone network of this invention first utilizes the W-MSA / SW-MSA mechanism of the Swing Transformer to model the contextual relationships in the complex environment under the vehicle, then enhances the local texture and fault details of bolts through DWC-RepConv, ultimately achieving a unified approach to global semantic modeling, local feature enhancement, and lightweight inference. This module can provide more stable, fine-grained, and discriminative visual features for subsequent TEAM-P1 dynamic fusion modules and detection heads.

[0049] 2. Improvements to the neck network of the TEAM-P1 feature fusion module This part is a neck network with the TEAM-P1 feature fusion module, which is used to solve the problems that small targets under the vehicle bolts are prone to losing texture details, insufficient utilization of axial arrangement relationships, and difficulty in highlighting fault areas such as rust and cracks during feature fusion.

[0050] Building upon the existing intra-scale feature interaction and cross-scale feature fusion of the neck structure, this invention adds a TEAM-P1 module. It consists of three parts: a P1 high-resolution feature reconstruction unit, an axially sensitive OFP fusion unit, and a TEAM dynamic texture enhancement attention unit. Figure 3 As shown.

[0051] (1) P1 high-resolution feature reconstruction unit The TEAM-P1 module simultaneously receives shallow high-resolution features output from the backbone network. and high-level semantic features ,in Used to restore local texture information such as bolt edges, threads, and small rust areas. Used to provide global semantic and structural context information. Let the shallow features be: First, through Convolution expands the number of channels to 4C: in Indicates extended features, For convolution kernel weights, For bias terms, The ReLU activation function is used. Then, subpixel rearrangement is used to convert channel information into spatial information, resulting in high-resolution features. in, This represents the high-resolution features after reconstruction. These correspond to the height direction, width direction, and channel index, respectively. This represents the modulo operation. This process can recover fine-grained information such as bolt head contours, thread boundaries, fracture notches, and small rust spots without significantly increasing the computational load.

[0052] (2) Axially sensitive OFP fusion unit To accommodate the horizontally arranged and vertically connected bolts to the chassis, an axially sensitive dual-path structure was designed. The horizontal path enhances the connection between bolts in the same row, as shown in the following formula: in Indicates a horizontal path. This represents a depthwise separable convolution with a kernel size of 3×1. And intermediate-scale features are obtained through pooling: in Indicates intermediate scale features. This indicates a max pooling operation with a pooling window size of 2×2. Vertical paths are used to model the structural relationship between bolts and the frame: in Indicates a vertical path. This represents a depthwise separable convolution with a kernel size of 1×3. Then and Perform cross-scale fusion to generate axis-sensitive fusion features: in Indicates axially sensitive fusion characteristics. This indicates concatenation along the channel dimension. This represents the Sigmoid activation function. express convolution, For adaptive fusion weights, this structure can simultaneously retain information on the arrangement of bolts in the same row and the bolt-frame structural relationship.

[0053] (3) TEAM Dynamic Texture Enhanced Attention Unit After cross-scale fusion, the features are fed into the TEAM dynamic texture enhancement attention unit. This unit is used to enhance irregularly shaped fault areas such as rust, cracks, and fractures, and includes the following three branches: Long-range context branches employ dilated convolutions to expand the receptive field: in This indicates the output of the long-range context branch. This represents dilated convolution with a kernel size of 3×3 and a dilation rate of 3. Indicates the void ratio; The deformation-aware branch employs deformable convolution to adapt to irregular shapes such as corrosion boundaries, crack paths, and fracture gaps. First, the offset is predicted: in express convolution; Then deformable sampling is performed: in This represents the output of the deformation-sensing branch. Indicates the current sampling position. This represents the regular offset of the nth sampling point in a 3×3 convolution kernel. This represents the learned deformation offset. This represents the convolution weight of the nth sampling point.

[0054] Multi-scale texture branching is used to extract bolt fault textures at different scales: in This represents the output of multi-scale texture branches. This represents a deformable depthwise separable convolution with a kernel size of 3×3. This represents a deformable depthwise separable convolution with a kernel size of 5×5. Indicates an upsampling operation. This indicates the average pooling operation. This indicates element-wise addition. Finally, the three branches are merged using dynamic weights: in These are the weights corresponding to the three branches. This indicates the fusion characteristics of the three branches. This represents the Softmax activation function. For the Sigmoid activation function, This represents element-wise multiplication, with the last term being... Residual connections are added to enhance training stability. To reduce distributional differences when fusing Transformer and convolutional features, LN and lightweight residual connections are added to the output: Final output It also includes high-resolution bolt texture, axial alignment, and fault area attention features, which can be used as input for the RT-DETR decoder or subsequent detection head, thereby significantly improving the sensitivity of identifying faults such as loose, missing, broken, and corroded bolts under the vehicle.

[0055] 3. Multi-scale sensing enhancement detection head This invention maintains the end-to-end detection structure of the RT-DETR encoder and decoder unchanged, introducing a local high-resolution feature enhancement mechanism only in the detection head after the decoder output. The target query vector output by the RT-DETR decoder first generates an initial candidate box or reference box. Based on this candidate box, this invention extracts local region features from the high-resolution feature map output by TEAM-P1. This RoI perception process is only used to enhance the detection head's ability to perceive local edges, threads, rust spots, and fracture gaps. It does not change the RT-DETR end-to-end prediction framework, nor does it introduce the traditional two-stage candidate box generation process. The flowchart is as follows: Figure 4 As shown.

[0056] (1) RoI Perception Enhancement Unit Let the set of target query vectors output by the RT-DETR decoder be: Where N represents the number of queries and C represents the channel dimension. Let represent the i-th original target query vector, where i ∈ [1, N]. Since undercarriage bolts are typically small, relying solely on the decoder query vector can easily lead to insufficient representation of local edges, threads, and fault details. Therefore, this invention introduces an RoI perception enhancement mechanism. First, based on the i-th original target query vector... Generate reference candidate boxes: in, This represents the reference candidate box corresponding to the i-th target query. This represents the reference box prediction subnetwork. Then, the high-resolution feature map output from TEAM-P1... Extract the local region features corresponding to the reference candidate box: in, This represents the local high-resolution features of the i-th candidate region. This indicates a region feature alignment operation. Subsequently, the local region features are pooled and linearly mapped, and then compared with the original target query vector. The enhanced target query vector is obtained by fusion: in, Indicates global average pooling. and For learnable weight matrix, Represents a non-linear activation function. This represents the enhanced query vector after fusing local high-resolution features. This process enables the detection head to not only utilize the global semantic information output by the decoder, but also to perceive local fine-grained fault features near the bolt target, such as edges, threads, rust spots, fracture notches, and loose displacement.

[0057] (2) Locating the branch (learnable bounding box offset mechanism) In the localization branch, this invention uses the enhanced target query vector. As input, the base bounding box is first predicted by the bounding box regression subhead: in, The base bounding box represents the i-th bolt target. This represents the bounding box regression network. Then, learnable bounding box offsets are predicted using the offset prediction subhead: in, This represents the bounding box correction offset for the i-th bolt target. The final bounding box coordinates are: in, This represents the revised final bolt inspection frame. (This is achieved by introducing...) The model can adaptively correct the basic prediction box based on the actual outline of the bolt target, local texture features and fault edge changes, thereby reducing the boundary offset error in small target detection and improving the positioning accuracy under fault conditions such as missing, broken, loose and corroded bolts.

[0058] (3) Classification branch (residual attention semantic enhancement) In the semantic classification branch, this invention introduces a residual attention mechanism to enhance the model's response to fault-sensitive channels. For the enhanced target query vector... First, calculate the attention weights for the classification channels: in, and For learnable weights, Represents the ReLU activation function. This represents the Sigmoid activation function. This represents the classification channel attention weight for the i-th target. Subsequently, the classification channel attention weights are multiplied channel-by-channel with the enhanced query vector, and the classification enhancement features are obtained through residual connections. in, This indicates channel-by-channel multiplication. This represents the classification enhancement feature for the i-th target. Finally, the classification enhancement feature is fed into the classification sub-header to obtain the probability distribution of bolt fault categories: in, Indicates the category subheading, Let represent the probability distribution of the i-th bolt target belonging to each fault category, which can be specifically expressed as: in Indicates normal probability. Indicates the probability of loosening. Indicates the probability of missing or detached items. This indicates the probability of breakage or incompleteness. This indicates the probability of corrosion.

[0059] Based on visual category probability distribution This yields the visual prediction category and visual confidence level: in, This represents the visual prediction category of the i-th bolt target. Let represent the visual confidence level of the i-th bolt target. Finally, the vision inspection module outputs the final detection bounding box for the i-th bolt target. Visual category probability distribution Visual prediction categories and visual confidence This output not only possesses extremely high local positioning accuracy but also keenly captures subtle surface fault features, serving as the standard input for subsequent visual-physical spatiotemporal mapping fusion determination mechanisms.

[0060] III. Spatiotemporal mapping fusion determination and output of visual inspection results and physical sensing nodes Based on the established spatial mapping table and the bolt detection results output by the visual inspection network, this invention constructs a spatiotemporal mapping fusion judgment mechanism between visual inspection results and physical sensing nodes. This mechanism no longer relies solely on visual inspection results for fault judgment; instead, through spatial mapping and temporal synchronization, it associates the visual inspection results of the same bolt area with corresponding physical state features and completes the fusion judgment. Figure 5 As shown.

[0061] 1. Deployment of physical sensor nodes and signal acquisition Physical sensing nodes can be deployed at key bolt groups, connecting plates, frame beams, bracket connections, equipment mounting bases, or easily loosened mounting holes on the bottom of the vehicle, and a binding relationship can be established with the corresponding bolt detection area according to the spatial mapping table.

[0062] Each physical sensing node may include one or more of a vibration sensor, a temperature sensor, and a continuity detection circuit. The vibration sensor is used to collect vibration signals caused by loose bolts or structural abnormalities; the temperature sensor is used to collect temperature change information in the connection area; for bolt areas with electrical connections or that can form a detection loop, continuity, resistance, or impedance detection circuits can be set to assist in determining conditions such as contact failure, detachment, or severe corrosion.

[0063] The physical state signal collected by the j-th physical sensor node at time t can be expressed as: in, Indicates vibration signal, Represents a temperature signal. This represents the conduction state, resistance, or impedance-related signal. After filtering and feature extraction, the physical feature vector of the j-th physical sensing node can be expressed as: in, It can include characteristics such as the root mean square value of vibration, peak value, dominant frequency, and frequency domain energy. It can include features such as current temperature, amount of temperature change, and rate of temperature change. Features can include conduction status, resistance change, and impedance change. The system can use IIR low-pass filtering, moving average filtering, or Kalman filtering to reduce noise in the original signal, thereby minimizing the impact of vehicle vibration, environmental noise, and acquisition jitter on the judgment results.

[0064] 2. Visual inspection result input and spatial mapping On the visual inspection side, the detection result of the i-th bolt target is improved by outputting RT-DETR: in, Indicates the timestamp of image acquisition or detection. This represents the final detection box. Represents the probability distribution of visual categories. Indicates the visual prediction category, This represents visual confidence. (Regarding visual confidence...) Greater than the set threshold For candidate targets, the system finds the corresponding physical sensor node based on the final detection box position: in, A spatial mapping function representing the visual detection box to the physical sensor node number. This represents the physical sensor node number corresponding to the i-th bolt target. Through this spatial mapping process, the system can accurately associate the visually detected bolt targets with their corresponding physical sensor nodes, avoiding erroneous matching between visual targets and irrelevant physical signals.

[0065] 3. Time synchronization and physical feature alignment After completing spatial mapping, the system uses visual detection timestamps. From the corresponding physical sensing node Extract physical signals within the same time window from cached data: And form the physical feature vector corresponding to the i-th bolt target: in, This represents the vibration characteristics extracted by the corresponding sensing node within this time window. Indicates temperature characteristics, It indicates the conduction state, resistance, or impedance-related characteristics.

[0066] 4. Visual-physical fusion determination Based on physical feature vectors The system calculates vibration anomaly scores, temperature anomaly scores, and continuity / impedance anomaly scores respectively: in, , , These represent scores for vibration, temperature, and continuity / impedance anomalies, respectively. , and These represent the anomaly scoring functions corresponding to the physical characteristics.

[0067] For each fault category y, calculate the fusion confidence score: in, ; This represents the fusion confidence score of the i-th bolt target belonging to category y; This represents the probability that the visual model predicts for category y. , , , and For the fusion parameters corresponding to the category, Indicates vibration characteristics, Indicates temperature characteristics, It indicates the conduction state, resistance, or impedance-related characteristics.

[0068] The final fusion decision category and the final fusion confidence level are as follows: in Indicates the fusion determination category, Indicates the final fusion confidence level. The independent variable representing the maximum value, This represents the maximum value.

[0069] When the visual prediction results are inconsistent with the physical state characteristics, the system can output a conflict status and prompt manual review. For example, if the visual prediction is normal but the physical anomaly score consistently exceeds the threshold, or if the visual prediction is loose but the vibration anomaly score is consistently below the threshold, both can be identified as visual-physical conflict.

[0070] 5. Results Output The output classification after fusion determination may include: Among them, normal means normal, loose means loose, missing means missing or detached, broken means broken or incomplete, rust means rusted, contact_failure means contact failure, and conflict means that the visual detection result is inconsistent with the physical perception result.

[0071] The final output can be represented as: in, This represents the final output result for the i-th bolt target. This indicates the alarm level. The system ultimately outputs the detection image and the bolt selection location. Visual prediction categories Fault Category Probability Visual confidence Corresponding sensor node number Physical signal waveform Fusion determination category Fusion confidence and alarm level The data is then uploaded to the inspection terminal or remote operation and maintenance platform.

[0072] Through the aforementioned mechanism, this invention does not simply splice visual results with physical signals. Instead, it first completes spatial mapping of sensor nodes based on visual detection locations, then synchronizes physical signals based on timestamps, and finally performs fusion judgment. This technical solution forms a complete closed loop of "visual detection positioning → sensor node mapping → time window synchronization → multi-source feature fusion → fault judgment output". Compared with single visual detection methods, this invention can use physical state signals to corroborate and correct visual detection results under conditions of insufficient lighting, oil stains, complex backgrounds, and unclear early fault visual features. Compared with simple multi-sensor fusion methods, this invention ensures an accurate correspondence between visual targets and physical signals through spatial mapping and time synchronization mechanisms, thereby improving the accuracy, robustness, and engineering deployability of undercarriage bolt fault detection.

[0073] In summary, the proposed under-vehicle bolt fault detection system, which combines visual and physical perception, has the following innovative advantages: (1) This invention proposes a spatiotemporal mapping fusion judgment mechanism for visual detection results and physical sensing nodes, which comprehensively judges the fault of bolts under the vehicle through "visual judgment, physical verification, and conflict review". Compared with a single visual detection method, this mechanism can use physical signals such as vibration, temperature and conduction status to corroborate the visual detection results under conditions such as uneven lighting, oil stains, and complex background under the vehicle, thereby reducing false detections and missed detections under a single visual modality.

[0074] (2) This invention does not simply stitch together visual data and sensor data, but establishes a spatial mapping relationship between the bolt target and the physical sensing node based on the position of the visual detection box, and extracts the physical signal within the corresponding time window based on the image acquisition timestamp. Through this spatiotemporal correspondence mechanism, each visual detection target can be matched with the physical state of the corresponding area, providing a spatial and temporal correspondence basis for multimodal fusion judgment.

[0075] (3) To address the issues of small target size, weak texture, and susceptibility to interference from complex backgrounds, this invention employs a lightweight improved RT-DETR network adapted for small target detection. This network, through the embedding of a lightweight Swin Transformer backbone network of DWC-RepConv, balances global context modeling and local texture enhancement capabilities, enabling the extraction of fine-grained features such as bolt edges, threads, rust spots, and fracture notches.

[0076] (4) This invention designs a TEAM-P1 feature fusion module and a multi-scale perception enhancement detection head to improve the visual feature representation capability of bolt faults. The TEAM-P1 module restores the texture and edge information in shallow high-resolution features through P1 super-resolution reconstruction, models the spatial arrangement relationship between bolts in the same row through axial sensitive fusion, and highlights fault-sensitive areas such as rust and cracks through dynamic texture enhancement attention; the detection head fuses high-resolution local features with decoder query vectors through RoI perception mechanism, adaptively corrects the initial prediction box through learnable anchor point offset, and enhances the response of fault-related channels through residual attention mechanism.

[0077] (5) This invention balances detection accuracy and engineering deployment requirements. The visual inspection network adopts a lightweight structural design, suitable for deployment on mobile inspection robots, under-vehicle inspection equipment, or vehicle-mounted edge computing platforms; the physical sensing part uses low-power sensor signals such as vibration, temperature, and conduction status, which can be acquired and preliminarily processed by embedded platforms such as STM32. The final system can output the detection image, bolt selection position, corresponding sensor node number, physical signal waveform, fusion judgment category, fusion confidence level, and alarm level, facilitating remote early warning and on-site maintenance.

[0078] Example 2 This invention proposes a fault detection method for under-vehicle bolts that combines visual and physical perception. The overall process is as follows: Figure 6 As shown, this method obtains the position, visual fault category, and visual confidence level of the bolts under the vehicle by improving the RT-DETR visual inspection network. Then, it matches the corresponding physical sensor nodes according to the position of the detection box, and combines the vibration, temperature, and conduction status signals within the same time window for fusion judgment. Finally, it outputs the bolt fault category, fusion confidence level, and alarm information.

[0079] like Figure 6 As shown, the specific implementation process of the present invention includes the following steps: S1: Acquire images of the vehicle's underside through the image acquisition module, and collect physical state signals of the physical area under the vehicle through physical sensor nodes; Images of the vehicle's underside are captured using industrial cameras or high-definition cameras mounted on mobile inspection robots, under-vehicle inspection equipment, or fixed inspection platforms. The captured images are then processed through size normalization, brightness enhancement, noise suppression, and pixel normalization before being used as input to the visual inspection network.

[0080] Physical sensing nodes are deployed in the key bolt areas under the vehicle. These nodes include vibration acceleration sensors, temperature sensors, and continuity detection modules. The vibration sensors detect minute vibration anomalies caused by bolt loosening. The temperature sensors assist in identifying abnormal friction, increased contact resistance, localized heat loss, and serve as auxiliary reference signals for rust or corrosion-related faults. For bolt areas with electrical connections or that can form detection loops, continuity detection modules can be installed to collect information on continuity, resistance, or impedance changes, helping to determine contact failure, detachment, or severe corrosion.

[0081] S2: The visual inspection module processes the image of the vehicle's bottom and outputs the bolt target's location detection box, visual fault category, and visual confidence level. The preprocessed undercarriage image is input into an improved RT-DETR network. This network includes a lightweight Swin Transformer backbone network embedded with DWC-RepConv, a TEAM-P1 feature fusion module, and a multi-scale perception-enhanced detection head. The visual detection module outputs the bounding box location, visual fault category, and visual confidence score for each bolt target.

[0082] S3: The spatial mapping module searches for the corresponding physical sensor node number in the preset spatial mapping table according to the position detection box; the time synchronization module extracts the physical state signal collected by the corresponding physical sensor node within the same time window according to the acquisition timestamp of the vehicle bottom image to obtain the physical feature vector. The region to which the bolt target belongs is determined based on the location of the visual inspection frame, and the corresponding physical sensor node number is found through a preset spatial mapping table. Subsequently, based on the image detection timestamp, vibration, temperature, and conduction status signals within the same time window are extracted from the cached data of the corresponding sensor node, achieving time synchronization between the visual inspection results and the physical sensing signals.

[0083] S4: The fusion determination module calculates the fusion confidence of each fault category based on the visual fault category, the visual confidence, and the physical feature vector, and determines the final bolt fault category; the result output module outputs the detection result containing the final bolt fault category. The visual detection category, visual confidence level, and physical state characteristics of the corresponding sensor node are input into the fusion judgment module. If the visual detection result is consistent with the physical state, the confidence level of the corresponding fault category is increased; if the visual result is inconsistent with the physical signal, a conflict or suspected fault indication is output. Finally, the detection image, bolt selection position, physical signal waveform, fusion judgment category, fusion confidence level, and alarm level are output and uploaded to the inspection terminal or remote operation and maintenance platform.

[0084] The under-vehicle bolt visual and physical perception fusion fault detection system in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network attached storage (NAS), personal computers (PCs), etc. This application embodiment does not impose specific limitations.

[0085] The under-vehicle bolt visual and physical perception fusion fault detection system in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0086] This application provides a fault detection system for undercarriage bolts that combines visual and physical perception, capable of achieving... Figure 1 The various processes implemented in the method embodiment of the visual and physical perception fusion fault detection method for under-vehicle bolts are not described in detail here to avoid repetition.

[0087] Optionally, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described embodiment of a method for detecting under-vehicle bolts by fusion of visual and physical perception, and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0088] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiment of a method for detecting under-vehicle bolts by fusion of visual and physical perception, and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0089] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0090] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0091] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0092] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A fault detection system for under-vehicle bolts that combines visual and physical perception, characterized in that, include: The image acquisition module is used to acquire images of the underside of the vehicle; Physical sensing nodes are used to collect physical state signals of the physical area under the vehicle; The visual inspection module is used to process the image of the bottom of the vehicle and output the location detection box of the bolt target, the visual fault category and the visual confidence level. The visual detection module incorporates an improved RT-DETR network, which includes: a backbone network for extracting local texture and shallow spatial features from the vehicle bottom image and outputting basic features; wherein the backbone network is a Swing Transformer network with an embedded DWC-RepConv composite convolutional enhancement module; a TEAM-P1 feature fusion module for receiving the basic features and performing intra-scale interaction and cross-scale fusion to output enhanced features; and a multi-scale perception enhancement detection head for making predictions based on the enhanced features and outputting the bolt target's location detection box, visual fault category, and visual confidence score. The backbone network comprises four hierarchical feature extraction stages. In the improved SwinTransformer Block of each feature extraction stage, the DWC-RepConv composite convolution enhancement module is embedded in the local window attention calculation output and before the MLP feedforward layer. During the model training stage, the DWC-RepConv composite convolution enhancement module enhances the input features through parallel depthwise separable convolution branches, reparameterized convolution branches, and identity mapping branches. During the model inference stage, the branches are equivalently fused into a single convolution for feature output. The TEAM-P1 feature fusion module includes: a P1 high-resolution feature reconstruction unit, used to perform channel expansion and sub-pixel rearrangement on the shallow features in the basic features to restore local high-resolution features; an axially sensitive OFP fusion unit, used to model and fuse the structural relationship between bolts in the same row and the frame using horizontal and vertical paths respectively, and output axially sensitive fused features; and a TEAM dynamic texture enhancement attention unit, used to process the axially sensitive fused features through dilated convolution, deformable convolution and multi-scale texture branching and fuse them based on dynamic weights, and output the enhanced features. The multi-scale perception enhancement detection head includes: a RoI perception enhancement unit, used to generate reference candidate boxes based on the target query vector output by the decoder, extract local region features from the enhancement features based on the reference candidate boxes and fuse them with the original target query vector, and output the enhanced target query vector; a localization branch, used to predict basic bounding boxes based on the enhanced target query vector, and predict learnable bounding box offsets to adaptively correct the basic bounding boxes, and output the location detection box; and a classification branch, used to calculate the classification channel attention weights using a residual attention mechanism and enhance the target query vector, and output the visual fault category and visual confidence. The spatial mapping module is used to find the corresponding physical sensor node number in a preset spatial mapping table based on the position detection box. The time synchronization module is used to extract the physical state signals collected by the physical sensing nodes within the same time window based on the timestamp of the vehicle bottom image, and obtain the physical feature vector. The fusion determination module is used to calculate the fusion confidence of each fault category based on the visual fault category, the visual confidence, and the physical feature vector, and determine the final bolt fault category. The results output module is used to output the detection results, which include the final bolt fault category.

2. The under-vehicle bolt visual and physical perception fusion fault detection system according to claim 1, characterized in that, The physical sensing node includes at least one of a vibration sensor, a temperature sensor, and a continuity detection circuit. The physical feature vector includes at least one of the following: vibration anomaly features, temperature anomaly features, conduction state, and resistance or impedance anomaly features extracted by filtering the physical state signal.

3. The under-vehicle bolt visual and physical perception fusion fault detection system according to claim 1, characterized in that, The specific determination process of the fusion determination module is as follows: Based on the physical feature vector, the vibration anomaly score, temperature anomaly score, and conduction / impedance anomaly score are calculated respectively. For each fault category, the predicted probability of the visual fault category and the anomaly score are weighted and summed using preset fusion parameters to obtain the fusion confidence of the fault category. The fault category corresponding to the maximum fusion confidence level is taken as the final bolt fault category.

4. A method for detecting under-vehicle bolts using a fusion of visual and physical perception, applied to the under-vehicle bolt visual and physical perception fusion fault detection system as described in any one of claims 1 to 3, characterized in that... Includes the following steps: The image acquisition module acquires images of the bottom of the vehicle, and the physical state signals of the physical area under the vehicle are collected by the physical sensor nodes. The vehicle underside image is processed by a visual inspection module, which outputs the bolt target location detection box, visual fault category, and visual confidence level. The spatial mapping module uses the position detection box to find the corresponding physical sensor node number in the preset spatial mapping table. The physical feature vector is obtained by extracting the physical state signals collected by the physical sensor nodes within the same time window based on the acquisition timestamp of the vehicle bottom image by the time synchronization module. The fusion determination module calculates the fusion confidence of each fault category based on the visual fault category, the visual confidence, and the physical feature vector, and determines the final bolt fault category. The results output module outputs the detection results, which include the final bolt fault category.

5. The method for fault detection of under-vehicle bolts by fusion of visual and physical perception according to claim 4, characterized in that, Before looking up the physical sensor node number, the process also includes: During the system initialization phase, the spatial mapping table is established. The spatial mapping table includes the bolt detection area number, the corresponding physical sensor node number, and the mapping relationship of the position range of the area in the vehicle under coordinate system.

6. The method for fault detection of under-vehicle bolts by fusion of visual and physical perception according to claim 4, characterized in that, The determination steps of the fusion determination module also include: Determine whether the visual fault category is consistent with the abnormal score of the physical feature vector. If the visual prediction is normal but the abnormal score exceeds the threshold, or the visual prediction is faulty but the abnormal score is below the threshold, output a visual-physical conflict status indicator and trigger a manual review prompt.

Images