Fault detection method, device and computer equipment

Abstract

The application relates to a fault detection method, device and computer equipment. The method comprises the following steps: a non-wearable terminal acquires field video data of a to-be-detected device; a device visual feature in the field video data is extracted; in the case that it is determined that the to-be-detected device has a target fault type according to the device visual feature and a defect knowledge graph, an operation guiding instruction corresponding to the target fault type is displayed; wherein the defect knowledge graph comprises standard visual features corresponding to different fault types; in response to a touch operation performed based on the operation guiding instruction, a front-end inspection device is controlled to detect the to-be-detected device according to a cooperation protocol between the non-wearable terminal and the front-end inspection device; and a fault detection result of the to-be-detected device is determined according to real-time detection data collected by the front-end inspection device in the detection process. The method can improve the operation efficiency and detection accuracy of remote fault review.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a fault detection method, apparatus and computer equipment. Background Technology

[0002] Remote inspection of power equipment is an important way to ensure the safe and stable operation of the power grid. At present, most mainstream remote inspection solutions rely on video equipment to collect on-site images, and then maintenance personnel remotely observe the equipment status and diagnose faults.

[0003] Existing technologies have several shortcomings: First, traditional inspection and maintenance personnel need to rely on their personal experience to identify inspection points and troubleshoot faults, which can easily lead to omissions in inspection paths and failure to check historical fault points, resulting in poor inspection standardization. Second, fault identification is highly dependent on the professional experience of personnel, and different personnel may have inconsistent identification results and handling procedures for the same type of fault, making it difficult to guarantee the accuracy of fault identification. Third, existing remote control methods have cumbersome operation procedures, requiring multiple switching of the operation interface to switch between functions such as temperature measurement, zoom, and historical image comparison, which cannot achieve quick linkage control and results in low inspection efficiency.

[0004] Meanwhile, some inspection solutions that use AR guidance generally require staff to wear AR glasses or other wearable devices. Wearable devices have drawbacks such as heavy wearing burden, poor adaptability to outdoor working conditions, and high hardware costs, making them unsuitable for long-term, large-scale remote inspection work. Summary of the Invention

[0005] Therefore, it is necessary to provide a fault detection method, apparatus, and computer equipment to address the aforementioned technical problems.

[0006] Firstly, this application provides a fault detection method, including:

[0007] Acquire on-site video data of the equipment to be tested;

[0008] Extract the device visual features from the on-site video data;

[0009] If, based on the device's visual features and the defect knowledge graph, it is determined that the device under test has a fault of the target fault type, the operation guidance instructions corresponding to the target fault type are displayed; wherein, the defect knowledge graph includes standard visual features corresponding to different fault types;

[0010] In response to a touch operation executed based on the operation guidance instruction, the front-end inspection device is controlled to inspect the device to be inspected according to the collaboration protocol between the non-wearable terminal and the front-end inspection device;

[0011] Based on the real-time detection data collected by the front-end inspection equipment during the detection process, the fault detection result of the equipment under test is determined.

[0012] Secondly, this application also provides a fault detection device, comprising:

[0013] The data acquisition module is used to acquire on-site video data of the equipment under test;

[0014] The feature extraction module is used to extract the visual features of the equipment from the on-site video data;

[0015] The instruction display module is used to display operation guidance instructions corresponding to the target fault type when it is determined that the device under test has a fault of the target fault type based on the device's visual features and the defect knowledge graph; wherein, the defect knowledge graph includes standard visual features corresponding to different fault types;

[0016] The device detection module is used to respond to a touch operation executed based on the operation guidance instruction, and control the front-end inspection device to detect the device to be detected according to the collaboration protocol between the non-wearable terminal and the front-end inspection device;

[0017] The result determination module is used to determine the fault detection result of the device under test based on the real-time detection data collected by the front-end inspection equipment during the detection process.

[0018] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0019] Acquire on-site video data of the equipment to be tested;

[0020] Extract the device visual features from the on-site video data;

[0021] If, based on the device's visual features and the defect knowledge graph, it is determined that the device under test has a fault of the target fault type, the operation guidance instructions corresponding to the target fault type are displayed; wherein, the defect knowledge graph includes standard visual features corresponding to different fault types;

[0022] In response to a touch operation executed based on the operation guidance instruction, the front-end inspection device is controlled to inspect the device to be inspected according to the collaboration protocol between the non-wearable terminal and the front-end inspection device;

[0023] Based on the real-time detection data collected by the front-end inspection equipment during the detection process, the fault detection result of the equipment under test is determined.

[0024] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0025] Acquire on-site video data of the equipment to be tested;

[0026] Extract the device visual features from the on-site video data;

[0027] If, based on the device's visual features and the defect knowledge graph, it is determined that the device under test has a fault of the target fault type, the operation guidance instructions corresponding to the target fault type are displayed; wherein, the defect knowledge graph includes standard visual features corresponding to different fault types;

[0028] In response to a touch operation executed based on the operation guidance instruction, the front-end inspection device is controlled to inspect the device to be inspected according to the collaboration protocol between the non-wearable terminal and the front-end inspection device;

[0029] Based on the real-time detection data collected by the front-end inspection equipment during the detection process, the fault detection result of the equipment under test is determined.

[0030] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0031] Acquire on-site video data of the equipment to be tested;

[0032] Extract the device visual features from the on-site video data;

[0033] If, based on the device's visual features and the defect knowledge graph, it is determined that the device under test has a fault of the target fault type, the operation guidance instructions corresponding to the target fault type are displayed; wherein, the defect knowledge graph includes standard visual features corresponding to different fault types;

[0034] In response to a touch operation executed based on the operation guidance instruction, the front-end inspection device is controlled to inspect the device to be inspected according to the collaboration protocol between the non-wearable terminal and the front-end inspection device;

[0035] Based on the real-time detection data collected by the front-end inspection equipment during the detection process, the fault detection result of the equipment under test is determined.

[0036] The aforementioned fault detection methods, devices, and computer equipment enable intelligent inspection operations based on ordinary non-wearable terminals, significantly reducing the hardware threshold and operational difficulty of remote inspection. By constructing a standardized fault feature and operation system through a defect knowledge graph, it solves the shortcomings of traditional manual inspection, which relies on personal experience, has inconsistent identification standards, non-standard operating procedures, and a high rate of false positives and false negatives, thus achieving digital, standardized, and intelligent fault identification. At the same time, through a collaborative protocol, multiple front-end devices such as high-definition shooting, infrared temperature measurement, and historical image comparison can be linked through screen touch operation without the need for complex operation switching, greatly improving the operational efficiency and detection accuracy of remote fault review. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a diagram illustrating the application environment of a fault detection method in one embodiment;

[0039] Figure 2 This is a flowchart illustrating a fault detection method in one embodiment;

[0040] Figure 3 This is a flowchart illustrating the process of determining a target fault type in one embodiment.

[0041] Figure 4 This is a schematic diagram of the process for testing the device under test in one embodiment;

[0042] Figure 5 This is a schematic diagram illustrating the process of displaying 3D images in one embodiment;

[0043] Figure 6 This is a flowchart illustrating visual guide elements in one embodiment;

[0044] Figure 7 This is a flowchart illustrating the process of determining spatial positioning results in one embodiment;

[0045] Figure 8 This is a flowchart illustrating the process of determining the priority of the current inspection operation in one embodiment;

[0046] Figure 9 This is a structural block diagram of a fault detection device in one embodiment;

[0047] Figure 10This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] The fault detection method provided in this application embodiment can be applied to, for example, Figure 1 The application environment is shown. Among them, non-wearable terminal 101 refers to a visual touch terminal that does not require user wearing and has no wearable hardware dependency, but is not limited to conventional display and control devices such as industrial touch screens, tablet terminals, and desktop touch monitors. This is distinct from wearable devices such as AR glasses, VR headsets, and motion-sensing wearable sensors. The device to be tested 102 specifically refers to high-voltage primary equipment in substations, including but not limited to high-voltage bushings, equipment oil tanks, coolers, gas relays, and other core power equipment prone to oil leakage, dust accumulation, overheating, and abnormal oil levels. The front-end inspection equipment 103 is a cluster of intelligent inspection equipment deployed on-site, including but not limited to high-definition zoom cameras, controllable infrared temperature measuring gimbals, and white light supplementary lighting gimbals, respectively responsible for detailed zoom shooting, area temperature scanning, and on-site lighting adjustment.

[0050] In one exemplary embodiment, such as Figure 2 As shown, a fault detection method is provided, which can be applied to... Figure 1 Taking non-wearable devices as an example, the specific steps include:

[0051] S201, Obtain on-site video data of the device under test.

[0052] Among them, the on-site video data is the visible light video stream data of the equipment collected in real time by the front-end high-definition camera equipment, including but not limited to the visual information of the real scene such as the equipment's appearance outline, surface color, light and shadow reflection, and component distribution.

[0053] Optionally, high-definition acquisition equipment deployed at the substation site can be used to acquire continuous visible light video streams of the power equipment under test in real time. Non-wearable terminals pull the on-site video data in real time via network protocols, completing video decoding, frame preprocessing, and synchronized image rendering to ensure that the terminal display is synchronized with the on-site scene in real time. Noise reduction and jitter removal preprocessing are performed on the video frames to eliminate invalid data caused by on-site lighting interference and image jitter, ensuring the accuracy of subsequent feature extraction.

[0054] For example, in routine inspection scenarios, the system defaults to real-time acquisition of video streams from all high-voltage equipment in the station, and polls to obtain on-site video data of each device to be inspected according to a preset inspection sequence. When the ambient temperature exceeds 30℃, the system automatically focuses on the high-voltage bushing equipment, prioritizing the acquisition of high-definition video data of the bushing area to achieve targeted acquisition of key equipment.

[0055] S202, Extract the visual features of the equipment from the on-site video data.

[0056] Among them, equipment visual features are the appearance features of the equipment extracted from the on-site video frames, including but not limited to oil stain reflection features, droplet distribution patterns, surface dust grayness, component outline deformation, observation window bubbles and blur features, etc.

[0057] Optionally, target region segmentation is performed on the preprocessed video frames to accurately extract the effective area of ​​the device to be detected, while eliminating invalid interference areas such as background structures, greenery, and sky. Visual feature extraction algorithms are used to extract the light and shadow features, texture features, morphological features, and color features of the device surface, quantifying and generating structured feature data that can be used for map matching. This eliminates the need for traditional subjective judgment by the human eye, achieving digital and standardized feature extraction.

[0058] For example, for the high-voltage bushing area, features such as bright reflective points, droplet outlines at the lower edge of the skirt, and oil stain texture are extracted to form a unique visual feature vector for oil seepage in the bushing. For the heat sink area of ​​the cooler, surface grayscale values ​​and texture uniformity features are extracted to identify abnormal visual information such as dust accumulation and dull color.

[0059] S203, if it is determined that the equipment under test has a fault of the target fault type based on the equipment's visual characteristics and defect knowledge graph, display the operation guidance instructions corresponding to the target fault type.

[0060] The defect knowledge graph is a pre-built structured knowledge base for power equipment inspection. It uses "equipment-fault" as the core entity and stores associated standard visual features, standardized operation guidance instructions, and fault judgment rules for various faults. This achieves accurate mapping between visual features and fault types and operation procedures, providing knowledge support for intelligent and standardized fault identification. The defect knowledge graph includes standard visual features corresponding to different fault types.

[0061] The target fault types are typical equipment faults identified through knowledge graph matching, including but not limited to high-frequency defect types of power equipment such as bushing oil leakage, equipment dust accumulation and overheating, false oil level in oil tank, and abnormal oil level in gas relay. The operation guidance instructions are standardized inspection operation steps preset for different fault types, including but not limited to specific operation guidelines such as supplementary light switch control, pan-tilt angle adjustment, zoom shooting, infrared temperature measurement, and historical image comparison, used to standardize the manual review process.

[0062] Optionally, intelligent initial fault assessment can be performed based on a pre-built defect knowledge graph, which stores a standard visual feature sample library for various faults. The real-time extracted device visual features are compared with the standard visual features of each fault type within the knowledge graph. When the similarity exceeds a preset threshold, the device is determined to have the corresponding target fault type. After fault determination, the standardized operating procedures associated with that fault type are automatically retrieved, and a dedicated operation guidance instruction is displayed in a pop-up window on the terminal display to guide the user in completing a precise review.

[0063] For example, when the matching degree of the reflective characteristics of the casing oil stains exceeds the standard, it is determined that there is a casing oil leakage fault, and the instruction pops up: "Please turn off the white light supplement, adjust the gimbal elevation angle to 45°±5°, and observe whether the oil droplets are distributed along the lower edge of the casing skirt." When the oil conservator observation window is blurred and contains bubble characteristics, it is determined that there is an abnormal oil level fault, and the instruction pops up: "Adjust the gimbal shooting angle at multiple angles to eliminate light reflection interference, and check the matching of the oil level scale with the ambient temperature."

[0064] S204, in response to a touch operation executed based on an operation guidance instruction, controls the front-end inspection device to inspect the device to be inspected according to the collaboration protocol between the non-wearable terminal and the front-end inspection device.

[0065] Touch operations are lightweight interactive operations performed by users on the display of non-wearable terminals, including long press, selection, and double-tap operations. The collaboration protocol is a pre-defined linkage mapping protocol between non-wearable terminals and front-end inspection devices, defining a one-to-one correspondence between different touch operations and device control commands, enabling a single touch command to drive multi-device collaborative operations.

[0066] Optionally, users can perform corresponding touch operations on the screen of a non-wearable terminal based on the operation guidance instructions displayed on the interface, with the touch operation type and touch location identified in real time. Based on the collaborative protocol, an automatic mapping between touch behavior and device instructions is established, eliminating the need for manual device switching and parameter input. It can link various front-end inspection devices with one click to complete targeted and refined inspections, achieving an upgrade from single-screen viewing to multi-device collaborative verification.

[0067] S205, based on the real-time detection data collected by the front-end inspection equipment during the inspection process, determines the fault detection result of the equipment to be inspected.

[0068] Among them, real-time detection data is structured data returned by the front-end inspection equipment after performing the inspection, including 20x zoom detail images, regional infrared temperature data, historical comparison images under the same working conditions, equipment attitude parameters, etc.

[0069] Optionally, the system aggregates multi-dimensional real-time detection data from the front-end inspection equipment, including zoom detail images, infrared temperature data, and historical comparison images. This data is then combined with the fault judgment criteria built into the defect knowledge graph for comprehensive analysis. By comparing the defect morphology in detail images, calculating the temperature difference in the calculated area, and comparing historical equipment status changes, the system comprehensively determines whether a fault truly exists, distinguishes the severity of the fault, and ultimately outputs standardized and traceable fault detection results, completing the fault detection closed loop.

[0070] For example, by combining the oil droplet distribution pattern in the 20x zoom image, infrared temperature difference data, and historical image changes, it can be determined whether the bushing has a persistent oil leakage fault. Based on the dust accumulation grayscale characteristics of the cooler and the infrared temperature difference gradient, it can be determined whether there is an overheating defect in the equipment caused by dust accumulation.

[0071] Among the aforementioned fault detection methods, intelligent inspection operations are achieved based on ordinary non-wearable terminals, significantly reducing the hardware threshold and operational difficulty of remote inspection. By constructing a standardized fault feature and operation system through a defect knowledge graph, the shortcomings of traditional manual inspection, such as reliance on personal experience, inconsistent identification standards, non-standardized operation procedures, and high rates of false positives and false negatives, are solved, realizing the digitalization, standardization, and intelligence of fault identification. At the same time, through a collaborative protocol, multiple front-end devices such as high-definition shooting, infrared temperature measurement, and historical image comparison can be linked through screen touch operation without the need for complex operation switching, greatly improving the operational efficiency and detection accuracy of remote fault review.

[0072] Optionally, in an exemplary embodiment, such as Figure 3 As shown, a method for determining a target fault type is provided, which specifically includes the following steps:

[0073] S301, determine the similarity between the standard visual features and the equipment visual features corresponding to different fault types.

[0074] The standard visual features are standard baseline feature vectors pre-stored in the defect knowledge graph, corresponding to each fault type. They are standardized feature templates trained on a large number of samples and manually verified. The standard visual features include fault-specific light and shadow features, texture features, morphological features, color features, and distribution features, which are used as comparison benchmarks for real-time fault identification.

[0075] Similarity is a quantitative value used to characterize the degree of matching and fit between real-time device visual features and standard visual features. It is calculated through feature space distance algorithm and feature vector matching algorithm. The higher the value, the closer the current device screen features are to the fault standard features.

[0076] Optionally, firstly, retrieve the standard visual feature templates corresponding to all pre-stored fault types in the defect knowledge graph, and simultaneously read the real-time device visual feature vector of the device to be detected extracted in the current frame. Map the two sets of features to the same feature space, and use a feature matching algorithm to calculate the difference between the standard features and real-time features for each fault type one by one, quantify and output the feature similarity value corresponding to each type of fault, and complete the parallel matching and comparison of multiple fault types.

[0077] In this embodiment, a multi-dimensional feature fusion comparison mechanism can be adopted, which is not a single pixel comparison, but a comprehensive calculation that integrates multiple features such as light and shadow, texture, shape and distribution. This effectively avoids mismatch problems caused by changes in lighting, slight image shaking, and shooting angle deviation, and improves the stability of feature comparison.

[0078] For example, the standard oil stain reflection features and oil droplet outline distribution features corresponding to oil seepage in the casing are retrieved from the knowledge graph; the reflective highlights and the texture wetting features of the lower edge of the umbrella skirt extracted from the casing are matched with the standard template, and the similarity of light and shadow, shape and distribution is calculated. The weighted average is then used to obtain the comprehensive similarity score of the oil seepage fault.

[0079] S302, if the similarity between any standard visual feature and the device visual feature is greater than the similarity threshold, then it is determined that the device to be tested has a fault of the target fault type.

[0080] The similarity threshold is a preset critical quantification value for fault determination. It is a fixed threshold calibrated through extensive field testing and is used to distinguish between normal equipment features and fault features. It can be configured independently for different equipment and different fault types. The target fault type is the fault type corresponding to the standard visual features with a similarity greater than the similarity threshold.

[0081] Optionally, the similarity values ​​corresponding to each type of fault are compared with preset similarity thresholds. If the similarity of any fault exceeds the threshold, the device is deemed to be abnormal, eliminating reliance on subjective human judgment. Simultaneously, the fault category corresponding to the visual feature exceeding the threshold is precisely defined as the target fault type for the current device.

[0082] In this embodiment, multiple fault identification for a single device is supported. If the similarity of multiple fault types exceeds the threshold at the same time, multiple defects can be determined simultaneously and corresponding operation instructions can be pushed in sequence, covering complex equipment abnormality scenarios.

[0083] For example, the preset similarity threshold for casing oil leakage is 85%. The calculated similarity of the current equipment oil leakage feature is 92%, which is greater than the threshold. It is determined that there is a fault in the high-pressure casing to be tested, and the target fault type is located as casing oil leakage fault. Then, a dedicated operation guidance instruction for oil leakage is matched and popped up.

[0084] In this embodiment, intelligent fault determination of equipment is achieved by using quantitative feature similarity comparison, which transforms the fault identification process into a standardized, quantifiable, and reproducible numerical matching process, greatly reducing the probability of fault misjudgment and missed judgment. Through the parallel matching and comparison mechanism of multiple fault types, all preset defect types can be traversed at once to achieve accurate identification and classification of multiple faults in a single device, ensuring the uniqueness and accuracy of fault location.

[0085] Optionally, in an exemplary embodiment, such as Figure 4 As shown, a method for testing a device to be tested is provided, which specifically includes the following steps:

[0086] S401, in response to a touch operation executed based on an operation guidance instruction, determines the operation type and operation content corresponding to the touch operation.

[0087] Touch operations include long press, selection by bounding box, and double tap. A long press refers to a user continuously pressing on the faulty area of ​​the terminal screen for more than a preset duration threshold; a selection by bounding box refers to a user manually drawing a rectangular selection area on the terminal screen to define the faulty area; and a double tap refers to a user quickly tapping the faulty point on the screen twice.

[0088] Optionally, after displaying the operation guidance instructions corresponding to the fault on the interface, the touch events on the terminal screen are monitored in real time. When the user completes the screen touch operation according to the guidance instructions, the touch behavior data is captured in real time. The touch logic algorithm distinguishes the current operation as any of the operation types such as long press, selection, and double tap. At the same time, the operation content corresponding to this touch is accurately analyzed, including key information such as touch pixel coordinates, effective operation area range, press duration, and click timing parameters.

[0089] For example, in the operation guidance instructions for casing oil leakage faults, the user long presses the fault area on the screen, and if the press duration reaches a preset threshold of 3 seconds, the operation type is determined to be a long press operation; at the same time, the coordinates of the long press center point are recorded as the equipment fault location, and the corresponding location operation content parameters are generated.

[0090] For operation guidance instructions targeting localized overheating faults in equipment, the user selects the area of ​​abnormal heat in the equipment, the trajectory is identified as a rectangular selection action, and the operation type is determined to be a selection operation; at the same time, the coordinates of the upper left and lower right corners of the selected area are analyzed to determine the operation content of the local area to be measured.

[0091] For the operation guidance instructions for reviewing minor defects in equipment, the user double-clicks on the fault details, identifies the rapid double-click timing characteristics, determines the operation type as a double-click operation, records the precise coordinates of the double-click, and determines the target location to be zoomed in for shooting.

[0092] S402 determines the control commands for the front-end inspection equipment based on the operation type and content corresponding to the touch operation, as well as the collaboration protocol.

[0093] The operation content refers to the specific and effective parameter information corresponding to the touch operation, including but not limited to the control coordinate position, the range of the selected area, the press duration, and the click point, which are used to accurately locate the part to be detected and ensure that the device control accurately corresponds to the fault area.

[0094] Optionally, a collaborative protocol can be invoked, which internally establishes a binding mapping relationship between "operation type + operation content" and device control commands. Using the identified operation type as a matching index, combined with the location, region, and duration parameters in the operation content, the corresponding device execution action is accurately matched, and standardized control commands that can be directly recognized by the front-end device are automatically generated, without the need for manual parameter configuration or manual switching of device function interfaces.

[0095] In an optional embodiment, if the touch operation type is a long press operation, a control command is generated based on the collaboration protocol to retrieve historical images of the area corresponding to the operation content collected by the front-end inspection device under the same operating conditions. Here, historical images under the same operating conditions refer to the standard image archives of the equipment collected and stored by the front-end inspection device at a historical inspection time consistent with the current equipment ambient temperature, operating load, and time period.

[0096] Optionally, when the user touch operation is identified as a long press, the duration of the long press is first verified to meet a preset threshold requirement. Simultaneously, the device area coordinates corresponding to this long press operation on the screen are accurately obtained, i.e., the area corresponding to the operation content. Based on the collaborative protocol, the dedicated instruction logic corresponding to the long press operation is matched to generate a historical image retrieval control instruction. The control instruction carries the current device number, operating parameters, and target area coordinates, and sends a data retrieval request to the front-end inspection device. After receiving the instruction, the front-end inspection device retrieves historical inspection image data matching the current operating condition from the device database, filters standard historical images under the same device, the same observation angle, and similar environmental conditions, and transmits the historical image data back to the non-wearable terminal. The non-wearable terminal displays and overlays the real-time on-site video footage and historical images on the same screen for comparison, assisting maintenance personnel in intuitively comparing changes in device status and determining whether there are any abnormalities such as new defects or defect expansion.

[0097] In another optional embodiment, if the touch operation type is a selection operation, a control command is generated based on a collaborative protocol to control the front-end inspection device to perform a temperature scan on the area corresponding to the operation content. The area corresponding to the operation content is the screen pixel area coordinates generated by the touch operation; a long press corresponds to the neighborhood of the pressed center point, a selection box corresponds to the closed selection area formed by dragging, and a double click corresponds to the clicked coordinate point area, representing the precise target area focused by the front-end device's action. Temperature scanning refers to the detection action of the front-end infrared inspection device performing infrared thermal imaging acquisition on a specified area, which can output data on the area's highest temperature, lowest temperature, average temperature, and temperature gradient distribution.

[0098] Optionally, when the user's touch operation is identified as a selection operation, the coordinate range of the closed selection area formed by the user's manual dragging is analyzed to determine the local area of ​​the device corresponding to the operation content. The non-wearable terminal generates a dedicated control command for infrared temperature scanning through a collaborative protocol, and encapsulates the pixel coordinates and spatial position parameters of the selected area into the command and sends it to the front-end inspection device. After receiving the command, the front-end infrared inspection device quickly calibrates the gimbal angle and infrared field of view, automatically focuses on the local area of ​​the device selected by the user, and performs full-coverage, high-density infrared temperature scanning of the limited area, eliminating the problems of data redundancy and poor targeting in traditional full-area scanning. After the collection is completed, the front-end device transmits the area temperature matrix, the highest temperature point, and temperature difference data back to the non-wearable terminal in real time. The non-wearable terminal generates a temperature heat map and a temperature measurement report to determine hidden faults such as local overheating, component aging, and poor contact.

[0099] In another optional embodiment, if the touch operation type is a double-click operation, a control command is generated based on the collaboration protocol to control the front-end inspection device to zoom and capture the area corresponding to the operation content at a preset magnification. Here, preset magnification zoom shooting refers to the action of the front-end high-definition camera device focusing, magnifying, and capturing high-definition images of the target area according to a fixed optical magnification preset by the system, preferably 20x optical zoom, to magnify the detailed textures and minute lesions of the device, enabling precise verification of subtle defects.

[0100] Optionally, when the user's touch operation is identified as a double-tap, the screen coordinates of the double-tap are precisely located to determine the target detail area of ​​the device corresponding to the operation. The non-wearable terminal generates a zoom shooting control command with a fixed preset magnification based on the collaborative protocol and sends it to the front-end high-definition camera device. After receiving the command, the front-end device activates the optical zoom mechanism, adjusts the lens to the preset zoom magnification, and performs precise focusing, image stabilization, and high-definition capture of the double-tap target area. This acquires detailed image data such as the device's surface micro-texture, tiny lesions, oil droplet details, and cracks, and transmits the high-definition detailed image back to the non-wearable terminal for display in real time. This enables refined verification of suspected defects and solves the problem of unclear or inaccurate judgments in ordinary distant images.

[0101] The above embodiments configure dedicated device linkage logic for three types of standardized touch operations: long press, selection box, and double tap. Relying on the collaborative protocol, a precise one-to-one mapping between touch intentions and device detection actions is achieved, thus constructing a lightweight, standardized, and highly adaptable human-machine collaborative inspection mechanism.

[0102] S403 sends control commands to the front-end inspection equipment.

[0103] Among them, the control command is a standardized device control message generated by the terminal based on the touch operation. It contains information such as the device action type, execution parameters, and target location, and is used to instruct the front-end inspection equipment to inspect the device to be inspected.

[0104] Optionally, the generated standardized control commands are encapsulated into protocol messages and sent to the corresponding front-end inspection equipment in real time via a stable field communication link. The front-end inspection equipment continuously listens for terminal commands, parses the command parameters upon receiving the commands, automatically executes the corresponding detection actions, completes the detailed verification and inspection of the equipment under inspection, and simultaneously collects real-time field detection data and sends it back to the terminal, forming a closed-loop operation process of touch interaction—equipment execution—data feedback.

[0105] In this embodiment, through standardized touch operation classification and a preset device collaboration protocol mapping mechanism, an intelligent transformation from manual touch interaction to automatic collaborative detection by front-end devices is achieved. Relying on simple long press, selection, and double-tap lightweight touch actions, refined detection tasks of different dimensions can be accurately triggered, greatly simplifying the remote inspection operation process and reducing the threshold of manual operation and the probability of misoperation. At the same time, through dual parameter parsing of operation type and operation content, it is ensured that the device control commands accurately correspond to the fault location and fault verification requirements, avoiding invalid detection operations.

[0106] Optionally, in one embodiment, such as Figure 5 As shown, a method for displaying 3D images is provided, which specifically includes the following steps:

[0107] S501, acquire environmental operating condition data of the environment in which the device under test is located.

[0108] Among them, environmental condition data refers to the real-time status parameter data of the working environment of the power equipment under test. It is a key external parameter that affects the equipment's operating status, the probability of failure, and the priority of inspection. It mainly includes multi-dimensional environmental and operating parameters such as ambient temperature, ambient humidity, on-site light intensity, equipment operating load, and real-time wind speed.

[0109] Optionally, environmental sensors and equipment backend operation monitoring systems deployed at the substation site can collect real-time environmental condition data around the equipment under test. This data undergoes cleaning, filtering, outlier removal, and normalization to eliminate transient interference and ensure the authenticity and validity of the operating condition data. Non-wearable terminals bind the processed environmental condition data to the corresponding equipment number, establishing a device-operating condition correspondence. Different operating condition parameters correspond to different inspection risk levels and operational priorities, enabling adaptive inspection guidance based on actual on-site conditions.

[0110] S502 displays visual guidance elements based on on-site video data and environmental condition data.

[0111] Among them, the visual guidance elements refer to the virtual AR guidance layer that is superimposed and rendered on the real-world video screen of the terminal. It is precisely aligned with the real scene and is used to assist maintenance personnel in completing inspection tasks in a standardized and visual manner. It includes 3D navigation arrows to guide the inspection path, defect boxes to mark the type of target fault, and operation instruction panels to display operation guidance instructions.

[0112] The 3D navigation arrow is a three-dimensional virtual arrow marker with spatial orientation guidance capabilities. It can dynamically adjust its pointing angle, distance marking, and display position according to the equipment's spatial location and work priority, guiding maintenance personnel on their inspection routes and equipment observation locations. The defect box is a virtual annotation box used to accurately delineate equipment fault locations and areas with a high incidence of historical faults. It can adaptively lock abnormal areas for identified target fault types, achieving visual highlighting of fault locations and preventing manual oversight or misreading. The operation instruction panel is a virtual interactive panel floating on the terminal screen, used to display standardized operation guidance instructions, review steps, and observation precautions for corresponding fault types, providing standardized operation guidance for user touch operation and equipment review work. Table 1 illustrates the visual guidance elements provided in this application's embodiments.

[0113] Table 1 Visual Guiding Elements

[0114] Element type Rendering method Function Description 3D navigation arrows Semi-transparent blue floating arrow Real-time guidance to the next checkpoint Defective highlight frame Red pulse flashing outline Marking areas with high historical defect rates Operation indicator panel Bottom Floating Toolbar Display device-specific operation commands

[0115] Optionally, based on on-site video data, the system first completes scene spatial feature matching and spatial positioning through a pre-registered AR navigation model to accurately obtain the three-dimensional spatial position of the equipment to be inspected. Simultaneously, it combines environmental condition data to determine the current equipment's inspection priority and key inspection areas. Based on the spatial positioning results and condition priorities, the system dynamically generates and adaptively adjusts the display position, style, level, and status of three types of visual guidance elements. This is then overlaid and rendered on the real-time video feed on the terminal display, achieving precise alignment between the virtual guidance elements and the actual equipment scene.

[0116] Among them, the 3D navigation arrow dynamically adjusts the direction and distance marking of the inspection path according to the spatial location of the equipment; the defect box accurately locks the abnormal part corresponding to the identified target fault type and realizes the high-brightness marking of the fault point; the operation instruction panel synchronously loads the standardized operation guidance instructions corresponding to the current fault, realizing the integrated visual auxiliary inspection of path guidance, defect marking and operation guidance.

[0117] For example, under normal operating conditions with normal temperature and load, 3D navigation arrows are rendered according to a preset standard inspection sequence to guide the routine inspection path; static defect boxes are overlaid by default for areas with high historical failure rates, and a routine inspection operation guidance panel is displayed. When high-temperature, high-load, or high-risk operating conditions are identified, the inspection priority of the corresponding equipment is increased, the 3D navigation arrows point to the high-risk equipment first, the defect boxes use a pulse flashing highlight style, and the operation instruction panel actively pops up a key review prompt, enhancing the inspection guidance effect of high-risk equipment. When the knowledge graph identifies target failure types such as casing oil leakage, a fixed defect box is accurately overlaid in the area of ​​the equipment corresponding to the failure, locking the abnormal point; at the same time, a dedicated operation command for oil leakage review is pushed to the operation instruction panel in real time, and the 3D navigation arrows automatically focus on the faulty equipment, realizing targeted failure review guidance.

[0118] In this embodiment, the inspection path is standardized using 3D navigation arrows, faults and high-risk points are accurately marked using defect boxes, and the inspection process is standardized using an operation instruction panel, freeing the inspection operation from heavy reliance on the experience of maintenance personnel. Simultaneously, the inspection priority and guidance display effects are dynamically adjusted based on environmental condition data, achieving a layered guidance mechanism of standardized guidance for routine conditions, enhanced guidance for high-risk conditions, and targeted guidance for fault scenarios. This significantly reduces the probability of missed or incorrect inspections, improves the standardization, targeting, and intelligence of remote inspection operations, and achieves lightweight AR visualization guidance throughout the process using non-wearable terminals. With low hardware costs and wide applicability, it possesses extremely high engineering application value.

[0119] Optionally, in an exemplary embodiment, such as Figure 6 As shown, a method for displaying visual guide elements is provided, which specifically includes the following steps:

[0120] S601, based on the on-site video data, determine the spatial positioning result of the device under test in its environment.

[0121] Among them, the spatial positioning result is the three-dimensional spatial position information of the device to be detected obtained by solving the preset spatial registration AR navigation model. It includes spatial parameters such as the device's relative coordinates, shooting distance, viewing angle and attitude, and the pixel range of the device area, which are used to determine the precise superposition position of the virtual guide elements.

[0122] Optionally, the system receives on-site video data collected from the front end, performs frame-by-frame feature extraction on the video frames, and extracts stable scene feature points, equipment outline features, and key landmark features. The extracted scene feature points are then matched and compared with a pre-constructed 3D spatial model of the substation scene to determine the current camera's shooting pose, shooting distance, and viewing angle parameters, establishing a mapping relationship between the video's 2D pixel coordinate system and the on-site 3D physical coordinate system. Based on this mapping relationship, the system accurately calculates the 3D spatial coordinates and pixel area of ​​each device under inspection in the on-site environment, ultimately outputting stable and high-precision equipment spatial positioning results, providing a spatial coordinate benchmark for subsequent precise virtual element alignment with equipment display.

[0123] S602, determine the priority of the current inspection operation based on environmental condition data.

[0124] Among them, the priority of inspection operations is a weighted level of inspection order obtained by quantifying the degree of risk of environmental conditions.

[0125] Optionally, the acquired environmental operating data is parsed to extract key parameters such as ambient temperature, equipment operating load, equipment operating time, and historical fault records. These parameters are then compared with preset risk thresholds to quantify the real-time fault risk coefficient of each piece of equipment. Based on the risk coefficient, all equipment to be inspected at the station is sorted into high, medium, and low priority levels for inspection operations. High-risk equipment has the highest priority and must be displayed and reviewed first; routine equipment has the next highest priority and is inspected according to standard procedures, achieving an intelligent scheduling logic of "the higher the risk, the higher the priority of inspection."

[0126] For example, when the ambient temperature exceeds 30°C and the equipment is operating at full load, the risk of leakage and overheating is determined to be significantly increased. This equipment is then set as the highest priority inspection target, triggering a guided visual display first. For equipment with historical defect records, based on operating data, the risk of recurrence is determined to be higher, automatically increasing the inspection priority of this equipment and prioritizing the marking of high-incidence defect areas. Under stable operating conditions with suitable environment and light equipment load, all equipment maintains its regular inspection priority according to a preset fixed sequence, conducting standardized rotational inspections.

[0127] S603, based on spatial positioning results and task priority, determines the element information of the visual guidance elements.

[0128] Among them, element information is a complete set of rendering parameters for visual guide elements, including dynamic parameters such as element display position, display level, display style, blinking state, annotation content, and display priority.

[0129] Optionally, based on the equipment spatial positioning results, the basic display coordinates and coverage areas of 3D navigation arrows, defect boxes, and operation indicator panels are determined. Simultaneously, considering equipment operation priorities, the display style, rendering level, highlight status, and annotation content of various visual elements are dynamically adjusted. High-priority devices correspond to a highlight flashing style, a top-level display, and detailed prompts; regular-priority devices correspond to a standard static display style. Ultimately, complete element information including position, style, level, and content is generated, providing precise parameter support for interface rendering.

[0130] For example, for high-temperature, high-load, and high-risk equipment, elements such as flashing red pulses on defect boxes, highlighted and bolded 3D navigation arrows, and actively popped-up operation indicator panels at the top are configured to enhance visual alerts. For ordinary priority equipment, static standard-style navigation arrows and defect boxes are configured, and the operation indicator panels are hidden by default, retaining only basic path guidance to ensure a clean and clear interface. For equipment areas with identified faults, based on accurate spatial positioning results, defect box coordinates that fit the size of the fault area are generated, and corresponding operation guidance text content is matched to achieve precise fault location marking and personalized guidance.

[0131] S604, based on element information, displays visual guide elements.

[0132] Optionally, non-wearable terminals read the established complete element information and open an independent rendering layer on top of the real-time video screen. Following the specified display position, style, and rendering level, they dynamically overlay and render 3D navigation arrows, defect boxes, and operation indicator panels. During rendering, the element positions are updated synchronously with changes in the video perspective and screen displacement, ensuring that the virtual guide elements always accurately fit the real device scene without offset, jitter, or misalignment. This ultimately forms an integrated visual inspection interface where the path is guideable, defects are visible, and operations are instructive.

[0133] For example, based on priority, 3D navigation arrows point sequentially to each device to be inspected, dynamically displaying the optimal inspection path and guiding maintenance personnel to complete the full-site inspection according to risk priority. Identified fault areas are highlighted with defect boxes in real time, clearly marking abnormal points for easy location by maintenance personnel. Matching the current fault type, the operation instruction panel automatically displays the corresponding standardized review operation steps, providing intuitive operational guidelines for user-controlled touch-based inspections.

[0134] In this embodiment, video spatial positioning technology is used to achieve precise virtual-real alignment between guiding elements and the actual equipment scene. Simultaneously, environmental condition data is combined to dynamically and adaptively adjust the priority of inspection operations, upgrading the visual guidance from a "fixed template display" to a "condition-adaptive intelligent display." By using both spatial positioning results and priority parameters to constrain the information of the visual elements, navigation guidance, defect marking, and operation prompts are made more aligned with the actual inspection needs on site, ensuring both the accuracy of the inspection location and the rationality of the inspection focus.

[0135] Optionally, in an exemplary embodiment, such as Figure 7 As shown, a method for determining the spatial positioning result of a device under test in its environment is provided, which specifically includes the following steps:

[0136] S701 extracts scene visual features from the on-site video data and compares the scene visual features with the scene spatial model of the environment in which the device under test is located to obtain the comparison results.

[0137] The scene visual features are stable feature information extracted from each frame of the on-site video, including equipment outline features, scene landmark features, edge features, texture features, and corner features. These features are unique and stable, unaffected by lighting or slight changes in viewing angle, and are used for scene matching and localization. The scene spatial model is a pre-built offline 3D digital model of the area to be inspected, storing the 3D coordinates, outlines, and standard visual features of all equipment, buildings, and landmarks on-site. It serves as the spatial benchmark template for the entire inspection environment. The comparison result is the output data obtained by matching the real-time extracted scene visual features with the built-in standard features of the scene spatial model. It includes information such as feature matching points, matching similarity, and feature offset, used to verify the correspondence between the video and the standard scene.

[0138] Optionally, the received live video data is preprocessed frame by frame to perform noise reduction, jitter reduction, and image enhancement. Then, a feature extraction algorithm is used to traverse the entire frame and select scene visual features with strong stability and high recognizability. Subsequently, the locally stored scene space model is called to perform global matching calculations between the real-time features and the pre-stored standard features in the model. Data such as the number of feature matches, the degree of matching overlap, and the feature offset distance are statistically analyzed to generate a quantitative comparison result, thereby determining the actual scene area corresponding to the current video frame.

[0139] For example, when the front-end camera is fixed in position and the scene remains unchanged, static visual features such as the device itself and its surrounding structure are extracted and quickly matched with standard features of the scene space model. If the similarity is higher than a preset threshold, a high-matching comparison result is output to confirm that the scene has not shifted. When the front-end gimbal rotates or the shooting angle changes, the content of the scene changes accordingly. The visual features of the new scene are extracted in real time and continuously compared dynamically with the scene space model. The feature offset is updated in real time, and a comparison result with position offset parameters is output to adapt to scenes with changing viewing angles.

[0140] S702 constructs a mapping relationship between the video pixel coordinate system and the three-dimensional physical coordinate system based on the video acquisition parameters of the on-site video data.

[0141] Among them, video acquisition parameters are the operating parameters of the front-end camera equipment when acquiring video, including lens focal length, shooting angle, gimbal attitude, image resolution, and field of view. These are the basic parameters for constructing coordinate mapping relationships. The video pixel coordinate system is a two-dimensional planar coordinate system with the upper left corner of the video screen as the origin and pixels as the unit, used to describe the position of equipment and feature points on the screen. The three-dimensional physical coordinate system is a three-dimensional spatial coordinate system with a selected reference point at the inspection site as the origin and physical length as the unit, used to describe the actual spatial position of the equipment to be inspected and the site landmarks. The mapping relationship is the conversion formula and conversion matrix between two-dimensional pixel coordinates and three-dimensional physical coordinates, which can realize the mutual conversion between the pixel position on the screen and the actual spatial position on site.

[0142] Optionally, video acquisition parameters such as focal length, field of view, gimbal tilt angle, and horizontal rotation angle reported by the front-end camera device are read. Combined with the camera imaging principle, the camera's intrinsic and extrinsic parameter matrices are calculated. Based on these parameters, a mathematical transformation model is established, creating a bidirectional mapping relationship between the two-dimensional video pixel coordinate system and the on-site three-dimensional physical coordinate system. This enables accurate conversion from pixel position to real-space coordinates and from spatial coordinates to image pixel position. This mapping relationship is updated synchronously with changes in camera device parameters, ensuring the coordinate transformation remains effective.

[0143] S703, based on the mapping relationship and comparison results, determine the spatial positioning result of the device under test in its environment.

[0144] Optionally, based on the obtained comparison results, the corresponding on-site area and feature offset of the current video frame are identified; then, relying on the constructed pixel-3D spatial mapping relationship, the pixel area where the device to be detected is located in the image is converted into spatial coordinates in a 3D physical coordinate system. At the same time, the coordinates are corrected and compensated according to the feature offset to eliminate positioning errors caused by viewpoint and image offset, and finally, the complete spatial positioning result of the device to be detected is output, including 3D coordinates, relative orientation, distance and other information.

[0145] In this embodiment, scene matching is achieved by using scene visual features and a pre-built scene spatial model, effectively resisting external interference such as lighting, image jitter, and viewpoint rotation, and improving the stability of scene recognition; a dual coordinate system mapping relationship is constructed based on video acquisition parameters to ensure the accuracy of coordinate transformation from the imaging principle level; and the positioning coordinates are corrected and compensated by combining the comparison results to further reduce positioning errors.

[0146] Optionally, in one embodiment, such as Figure 8 As shown, a method for determining the priority of the current inspection operation is provided, which specifically includes the following steps:

[0147] S801 determines the ambient temperature and equipment operating load parameters based on environmental condition data.

[0148] Among them, ambient temperature is the real-time air temperature of the area surrounding the equipment under test, which is collected by on-site temperature sensors. Equipment operating load parameters are numerical indicators characterizing the current operating power of electrical equipment, usually expressed as a percentage of rated load. The higher the load, the more significant the internal losses and temperature rise of the equipment, and the greater the risk of failure.

[0149] Optionally, the system receives aggregated environmental condition data, categorizes and filters it according to preset data fields, removes data noise, instantaneous outliers, and transmission errors, extracts valid ambient temperature values ​​and equipment operating load values, and binds and stores these two sets of parameters with the corresponding equipment to be tested. Simultaneously, the parameters are standardized to unify data format and units, ensuring that the parameters can be directly used for subsequent risk assessment and priority calculation.

[0150] S802 determines the priority of the current inspection operation based on the ambient temperature and equipment operating load parameters.

[0151] Optionally, ambient temperature thresholds, equipment load thresholds, and rules for their combination can be pre-configured. The extracted real-time ambient temperature and operating load are compared with their corresponding thresholds, and the combined rules are used to comprehensively assess the equipment failure risk level. Inspection operation priorities are then assigned based on the risk level. Higher temperatures and heavier operating loads indicate a higher equipment failure risk, and therefore a higher inspection operation priority is set; conversely, lower temperatures result in a normal priority. This process completes the priority ranking of all equipment at the station, providing a basis for subsequent differentiated visual guidance and inspection task scheduling.

[0152] In this embodiment, the priority of inspection operations is quantified and automatically divided based on two core parameters: ambient temperature and equipment operating load. By jointly assessing the risk of equipment operation through dual parameters, the priority judgment standard is unified and the logic is clear. It can accurately identify high-risk operating scenarios such as high temperature and high load, and automatically increase the inspection weight of high-risk equipment.

[0153] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0154] Based on the same inventive concept, this application also provides a fault detection device for implementing the fault detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more fault detection device embodiments provided below can be found in the limitations of the fault detection method described above, and will not be repeated here.

[0155] In one exemplary embodiment, such as Figure 9 As shown, a fault detection device 900 is provided, configured in a non-wearable terminal, including: a data acquisition module 910, a feature extraction module 920, an instruction display module 930, a device detection module 940, and a result determination module 950, wherein:

[0156] Data acquisition module 910 is used to acquire on-site video data of the device under test;

[0157] Feature extraction module 920 is used to extract the visual features of equipment from the on-site video data;

[0158] The instruction display module 930 is used to display operation guidance instructions corresponding to the target fault type when it is determined that the device under test has a fault of the target fault type based on the device's visual features and the defect knowledge graph; wherein, the defect knowledge graph includes standard visual features corresponding to different fault types;

[0159] The device detection module 940 is used to respond to touch operations based on operation guidance instructions and control the front-end inspection device to detect the device to be tested according to the collaboration protocol between the non-wearable terminal and the front-end inspection device.

[0160] The result determination module 950 is used to determine the fault detection result of the equipment under test based on the real-time detection data collected by the front-end inspection equipment during the detection process.

[0161] In one embodiment, the instruction display module 930 is used for:

[0162] Determine the similarity between the standard visual features and the device visual features corresponding to different fault types; if the similarity between any standard visual feature and the device visual feature is greater than the similarity threshold, then determine that the device under test has a fault of the target fault type; where the target fault type is the fault type corresponding to the standard visual feature with a similarity greater than the similarity threshold.

[0163] In one embodiment, the device detection module 940 includes:

[0164] The content determination unit is used to determine the operation type and operation content corresponding to the touch operation in response to the touch operation executed based on the operation guidance instruction; wherein the touch operation includes a long press operation, a selection operation, or a double-click operation;

[0165] The instruction determination unit is used to determine the control instructions for the front-end inspection equipment based on the operation type and operation content corresponding to the touch operation and the collaboration protocol.

[0166] The equipment detection unit is used to send control commands to the front-end inspection equipment; the control commands are used to instruct the front-end inspection equipment to perform inspection on the equipment to be inspected.

[0167] In one embodiment, the instruction determining unit is specifically used for:

[0168] If the touch operation type is a long press, a control command is generated based on the collaboration protocol to retrieve historical images of the area corresponding to the operation content collected by the front-end inspection device under the same working conditions. If the touch operation type is a box selection operation, a control command is generated based on the collaboration protocol to control the front-end inspection device to perform temperature scanning on the area corresponding to the operation content. If the touch operation type is a double-tap operation, a control command is generated based on the collaboration protocol to control the front-end inspection device to zoom and take pictures of the area corresponding to the operation content at a preset magnification.

[0169] In one embodiment, the fault detection device 900 further includes a screen display module, comprising:

[0170] The data acquisition unit is used to acquire environmental operating condition data of the environment in which the device under test is located.

[0171] The element display unit is used to display visual guidance elements based on on-site video data and environmental condition data. The visual guidance elements include 3D navigation arrows to guide the inspection path, defect boxes to mark the type of target fault, and operation instruction panels to display operation guidance instructions.

[0172] In one embodiment, the element display unit includes:

[0173] The result determination unit is used to determine the spatial positioning result of the device under test in its environment based on the on-site video data.

[0174] Priority determination unit is used to determine the priority of the current inspection operation based on environmental condition data;

[0175] Information is determined from the unit, which is used to determine the element information of the visual guidance element based on the spatial positioning results and the task priority;

[0176] The element display unit is used to display visual guide elements based on element information.

[0177] In one embodiment, the result determination from the unit is specifically used for:

[0178] The visual features of the scene are extracted from the on-site video data, and the visual features of the scene are compared with the scene spatial model of the environment in which the device under test is located to obtain the comparison results; and, based on the video acquisition parameters of the on-site video data, a mapping relationship between the video pixel coordinate system and the three-dimensional physical coordinate system is constructed; based on the mapping relationship and the comparison results, the spatial positioning result of the device under test in its environment is determined.

[0179] In one embodiment, priority determination from the unit is specifically used for:

[0180] Based on environmental condition data, determine the ambient temperature and equipment operating load parameters; based on the ambient temperature and equipment operating load parameters, determine the priority of the current inspection operation.

[0181] Each module in the aforementioned fault detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0182] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10As shown, the computer device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a fault detection method.

[0183] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0184] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method provided in the above embodiments.

[0185] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the above embodiments.

[0186] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method provided in the above embodiments.

[0187] It should be noted that the data involved in this application (including but not limited to data used for analysis, data stored, data displayed, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0188] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0189] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0190] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A fault detection method, characterized in that, Applied to non-wearable terminals, the method includes: Acquire on-site video data of the equipment to be tested; Extract the device visual features from the on-site video data; If, based on the device's visual features and the defect knowledge graph, it is determined that the device under test has a fault of the target fault type, the operation guidance instructions corresponding to the target fault type are displayed; wherein, the defect knowledge graph includes standard visual features corresponding to different fault types; In response to a touch operation executed based on the operation guidance instruction, the front-end inspection device is controlled to inspect the device to be inspected according to the collaboration protocol between the non-wearable terminal and the front-end inspection device; Based on the real-time detection data collected by the front-end inspection equipment during the detection process, the fault detection result of the equipment under test is determined.

2. The method according to claim 1, characterized in that, The step of determining whether the device under test has a target fault type based on the device's visual features and defect knowledge graph includes: Determine the similarity between the standard visual features corresponding to different fault types and the visual features of the equipment; If any standard visual feature has a similarity greater than a similarity threshold with the visual feature of the device, then the device to be detected is determined to have a fault of the target fault type; wherein, the target fault type is the fault type corresponding to the standard visual feature with a similarity greater than the similarity threshold.

3. The method according to claim 1, characterized in that, The step of responding to a touch operation based on the operation guidance instruction, and controlling the front-end inspection device to inspect the device under test according to the collaboration protocol between the non-wearable terminal and the front-end inspection device, includes: In response to a touch operation executed based on the operation guidance instruction, the operation type and operation content corresponding to the touch operation are determined; wherein, the touch operation includes a long press operation, a selection box operation, or a double-click operation; Based on the operation type and operation content corresponding to the touch operation, and the collaboration protocol, determine the control command for the front-end inspection equipment; The control command is sent to the front-end inspection device; wherein the control command is used to instruct the front-end inspection device to inspect the device to be inspected.

4. The method according to claim 3, characterized in that, The step of determining the control command for the front-end inspection device based on the operation type and operation content corresponding to the touch operation, and the collaboration protocol, includes: If the touch operation type is a long press operation, then based on the collaboration protocol, a control command is generated to retrieve the historical images of the same working condition in the area corresponding to the operation content collected by the front-end inspection device. If the touch operation type is a box selection operation, then based on the collaboration protocol, a control command is generated to control the front-end inspection device to perform temperature scanning on the area corresponding to the operation content; If the touch operation type is a double-click operation, then based on the collaboration protocol, a control command is generated to control the front-end inspection device to zoom and take pictures of the area corresponding to the operation content at a preset magnification.

5. The method according to claim 1, characterized in that, The method further includes: Obtain environmental condition data of the environment in which the device under test is located; Based on the on-site video data and the environmental condition data, visual guidance elements are displayed; wherein, the visual guidance elements include 3D navigation arrows for guiding the inspection path, defect boxes for marking target fault types, and operation instruction panels for displaying operation guidance commands.

6. The method according to claim 5, characterized in that, The step of displaying visual guidance elements based on the on-site video data and the environmental condition data includes: Based on the on-site video data, the spatial positioning result of the device under test in its environment is determined; Based on the environmental condition data, determine the priority of the current inspection operation; Based on the spatial positioning results and the task priority, determine the element information of the visual guidance elements; Based on the element information, the visual guide element is displayed.

7. The method according to claim 6, characterized in that, The step of determining the spatial positioning result of the device under test in its environment based on the on-site video data includes: Extract scene visual features from the on-site video data, and compare the scene visual features with the scene spatial model of the environment where the device under test is located to obtain the comparison result; and, Based on the video acquisition parameters of the on-site video data, a mapping relationship between the video pixel coordinate system and the three-dimensional physical coordinate system is constructed; Based on the mapping relationship and the comparison result, the spatial positioning result of the device under test in its environment is determined.

8. The method according to claim 6, characterized in that, The step of determining the priority of the current inspection operation based on the environmental condition data includes: Based on the environmental condition data, determine the ambient temperature and equipment operating load parameters; The priority of the current inspection operation is determined based on the ambient temperature and the equipment operating load parameters.

9. A fault detection device, characterized in that, Configured for use in non-wearable terminals, the device includes: The data acquisition module is used to acquire on-site video data of the equipment under test; The feature extraction module is used to extract the visual features of the equipment from the on-site video data; The instruction display module is used to display operation guidance instructions corresponding to the target fault type when it is determined that the device under test has a fault of the target fault type based on the device's visual features and the defect knowledge graph; wherein, the defect knowledge graph includes standard visual features corresponding to different fault types; The device detection module is used to respond to a touch operation executed based on the operation guidance instruction, and control the front-end inspection device to detect the device to be detected according to the collaboration protocol between the non-wearable terminal and the front-end inspection device; The result determination module is used to determine the fault detection result of the device under test based on the real-time detection data collected by the front-end inspection equipment during the detection process.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images