AR-based intelligent switch cabinet fault diagnosis method and device, and electronic equipment

Abstract

The invention discloses an AR-based intelligent switch cabinet fault diagnosis method and device and electronic equipment, and relates to the field of AR. The method comprises the following steps: acquiring a switch cabinet image collected by AR equipment; based on the switch cabinet image, identifying the internal structure of the switch cabinet by using a preset three-dimensional model identification library to obtain spatial position information of each component of the switch cabinet; constructing a virtual fault diagnosis environment of the switch cabinet according to the spatial position information of each component; acquiring a temperature distribution matrix of each component, and generating a thermodynamic cloud chart of each component according to the temperature distribution matrix; determining a problem part according to the thermodynamic cloud picture, and superposing the problem part to the virtual fault diagnosis environment to obtain a target fault diagnosis environment; and displaying the target fault diagnosis environment to the user through the AR equipment. According to the technical scheme provided by the invention, the internal environment and the problem parts of the switch cabinet are displayed to the user through the AR equipment, and the fault diagnosis efficiency is improved.

Description

Technical Field

[0001] The present application relates to the AR field, and specifically to an AR-based intelligent switch cabinet fault diagnosis method, device and electronic equipment. Background Art

[0002] With the rapid development of industrial automation and intelligent manufacturing, intelligent switchgear has become an indispensable part of power systems and industrial control systems. Intelligent switchgear integrates a variety of advanced functions, such as remote monitoring, automatic fault detection, and real-time data analysis to ensure the stable operation and safety of the system. The performance and reliability of the switchgear are directly related to the efficiency and safety of the entire power system. Therefore, timely and accurate fault diagnosis of intelligent switchgear is particularly important.

[0003] At present, the traditional switch cabinet fault diagnosis method mainly relies on regular manual inspection, which has cumbersome steps and low efficiency.

[0004] Therefore, there is an urgent need for an AR-based intelligent switch cabinet fault diagnosis method, device and electronic equipment. Summary of the invention

[0005] The present application provides an AR-based intelligent switch cabinet fault diagnosis method, device and electronic device, which displays the target fault diagnosis environment to the user through the AR device, thereby improving the efficiency of fault diagnosis.

[0006] In a first aspect of the present application, an AR-based intelligent switch cabinet fault diagnosis method is provided, the method comprising: obtaining a switch cabinet image captured by an AR device; based on the switch cabinet image, using a preset three-dimensional model recognition library to identify the internal structure of the switch cabinet, and obtain the spatial position information of each component of the switch cabinet; constructing a virtual fault diagnosis environment of the switch cabinet according to the spatial position information of each component; obtaining a temperature distribution matrix of each component, and generating a thermal cloud map of each component according to the temperature distribution matrix; determining the problem component according to the thermal cloud map, and superimposing the problem component on the virtual fault diagnosis environment to obtain a target fault diagnosis environment; and displaying the target fault diagnosis environment to the user through the AR device.

[0007] By adopting the above technical solution, by obtaining the switch cabinet image collected by the AR device, the internal structure of the switch cabinet is identified using the preset three-dimensional model recognition library, the spatial position information of each component is obtained, and a virtual fault diagnosis environment of the switch cabinet is constructed accordingly. At the same time, the temperature distribution matrix of each component is obtained, a thermal cloud map is generated, the problem component is determined and superimposed on the virtual fault diagnosis environment to form a target fault diagnosis environment, and finally displayed to the user through the AR device. This method combines AR technology with infrared thermal imaging to achieve a visual and immersive diagnosis experience of switch cabinet faults. Maintenance personnel can wear AR glasses to see the internal structure of the switch cabinet and the location, temperature and other information of the faulty components at a glance, which improves the efficiency and accuracy of diagnosis.

[0008] Optionally, based on the switch cabinet image, the internal structure of the switch cabinet is identified using a preset three-dimensional model recognition library to obtain spatial position information of each component of the switch cabinet, specifically including: extracting feature information of the switch cabinet image, and comparing the feature information with standard feature information in the preset three-dimensional model recognition library to obtain a comparison result; matching the three-dimensional model corresponding to the switch cabinet image according to the comparison result; reconstructing the internal structure of the switch cabinet according to the three-dimensional model to obtain spatial position information of each component of the switch cabinet.

[0009] By adopting the above technical solution, the feature information of the switch cabinet image is extracted and compared with the standard feature information to match the corresponding three-dimensional model, and the internal structure of the switch cabinet is reconstructed in three dimensions based on this to obtain the spatial position information of each component. This method realizes the rapid and automatic generation of the three-dimensional model of the switch cabinet. The accuracy of the three-dimensional reconstruction is guaranteed by the feature comparison and matching method, so that the three-dimensional model can truly reflect the internal structure of the switch cabinet.

[0010] Optionally, constructing a virtual fault diagnosis environment of the switch cabinet according to the spatial position information of each of the components specifically includes: generating a three-dimensional virtual model of each of the components based on the spatial position information of each of the components; acquiring working parameters of the switch cabinet, and associating the working parameters with the three-dimensional virtual models of each of the components to obtain a virtual fault diagnosis environment of the switch cabinet; wherein the working parameters include voltage, current and power.

[0011] By adopting the above technical solution, on the basis of obtaining the spatial position information of each component of the switch cabinet, a three-dimensional virtual model of each component is generated, and the working parameters of the switch cabinet are obtained. The working parameters are associated with the virtual component model to build a virtual fault diagnosis environment for the switch cabinet. This method not only realizes the three-dimensional visualization of the physical structure of the switch cabinet, but also integrates the key parameters reflecting its working status, so that the virtual fault diagnosis environment can comprehensively and dynamically simulate the actual operation of the switch cabinet. Maintenance personnel can monitor the working parameters of each component in real time in the virtual fault diagnosis environment, understand the health status of the equipment, and predict the possible types of faults.

[0012] Optionally, obtaining the temperature distribution matrix of each of the components specifically includes: controlling an infrared thermal imager to collect infrared thermal images of each of the components; performing image segmentation processing on the infrared thermal images to obtain temperature distribution areas of each of the components; extracting temperature values ​​of each of the temperature distribution areas to generate a temperature distribution matrix of each of the components.

[0013] By adopting the above technical solution, infrared thermal imaging technology is used to collect infrared thermal images of various components of the switch cabinet, and the temperature distribution area of ​​each component is obtained through image segmentation processing, and the temperature value is extracted to generate a temperature distribution matrix. This method realizes the synchronous acquisition and refined characterization of the temperatures of multiple components inside the switch cabinet. The temperature distribution matrix provides a data basis for subsequent temperature anomaly diagnosis and helps to accurately locate faulty components. At the same time, the acquisition of the temperature distribution matrix provides a quantitative basis for fault diagnosis, making the diagnosis process more objective and reliable.

[0014] Optionally, generating a thermal cloud map of each of the components according to the temperature distribution matrix specifically includes: normalizing the temperature distribution matrix to obtain a normalized temperature distribution matrix; constructing a thermal cloud map color scale, wherein the thermal cloud map color scale includes color identifiers corresponding to different temperature intervals; traversing each element in the normalized temperature distribution matrix to obtain a normalized temperature value of each element; determining a target color identifier in the thermal cloud map color scale corresponding to the normalized temperature value according to the normalized temperature value; in the virtual fault diagnosis environment, superimposing and rendering the target color identifier according to the position of each of the elements in the normalized temperature distribution matrix to obtain a thermal cloud map of each of the components.

[0015] By adopting the above technical solution, the temperature distribution matrix is ​​normalized, a thermal cloud map color scale is constructed, the normalized temperature value is mapped to the corresponding color, and the thermal cloud map of each component is generated by overlay rendering in the virtual diagnostic environment. This method intuitively and vividly displays the temperature distribution status of each component inside the switch cabinet, and reflects the temperature through color changes. Compared with the digital temperature matrix, the thermal cloud map is more intuitive and easy to understand, and maintenance personnel can quickly determine the abnormal area and change trend of the temperature. At the same time, overlay rendering on the three-dimensional virtual model makes the temperature information and the physical structure set off each other, which is more vivid and comprehensive. As a visual diagnostic tool, the thermal cloud map facilitates the identification and positioning of temperature anomalies and improves the efficiency and accuracy of diagnosis.

[0016] Optionally, determining the problem component according to the thermal cloud map and superimposing the problem component to the virtual fault diagnosis environment to obtain a target fault diagnosis environment specifically includes: calculating the average temperature value of each of the components according to the thermal cloud map; comparing the average temperature value of each of the components with a preset temperature threshold to obtain the problem component, wherein the problem component is a component having an average temperature value greater than or equal to the preset temperature threshold; in the virtual fault diagnosis environment, identifying the target position of the problem component, and superimposing the thermal cloud map corresponding to the problem component to the target position to obtain the target fault diagnosis environment.

[0017] By adopting the above technical solution, the average temperature value of each component is calculated according to the thermal cloud map, and by comparing with the preset temperature threshold, the components with excessive average temperature are screened out as problem components. The location of the problem component is identified in the virtual fault diagnosis environment, and the thermal cloud map is superimposed to form a target fault diagnosis environment. This method further quantifies the temperature level of the component on the basis of the thermal cloud map, and realizes the automatic identification and location of the problem component by comparing the preset temperature threshold. The thermal cloud map of the problem component is superimposed and displayed in the virtual fault diagnosis environment, making the fault location more eye-catching and the diagnosis focus more prominent. The target fault diagnosis environment integrates multi-dimensional information such as the physical structure of the switch cabinet, component location, temperature distribution, etc., providing an intuitive and reliable basis for fault cause analysis and maintenance decision-making.

[0018] Optionally, after acquiring the switch cabinet image captured by the AR device, the method also includes: judging whether the cabinet door of the switch cabinet is in an open state based on the switch cabinet image; if it is determined that the cabinet door of the switch cabinet is not in the open state, displaying a prompt message, and the prompt message is used to prompt the user to open the cabinet door of the switch cabinet.

[0019] By adopting the above technical solution, after acquiring the switch cabinet image, the image analysis is used to determine whether the cabinet door is open. If the cabinet door is not open, a prompt message is displayed to guide the user to open the cabinet door. This method introduces a human-computer interaction mechanism in the fault diagnosis process, and provides necessary prompts and interventions based on the actual working conditions. The opening state of the cabinet door directly affects the acquisition quality of the infrared thermal image and the reliability of the diagnosis. By promptly reminding the user to open the cabinet door, the consistency of the diagnostic environment and the actual operating conditions is ensured, and the credibility of the diagnostic results is improved.

[0020] In a second aspect of the present application, an AR-based intelligent switch cabinet fault diagnosis device is provided, which includes an acquisition module and a processing module; the acquisition module is used to acquire a switch cabinet image collected by an AR device; the processing module is used to identify the internal structure of the switch cabinet based on the switch cabinet image using a preset three-dimensional model recognition library to obtain the spatial position information of each component of the switch cabinet; the processing module is also used to construct a virtual fault diagnosis environment of the switch cabinet according to the spatial position information of each component; the acquisition module is also used to acquire the temperature distribution matrix of each component, and generate a thermal cloud map of each component according to the temperature distribution matrix; the processing module is also used to determine the problem component according to the thermal cloud map, and superimpose the problem component on the virtual fault diagnosis environment to obtain a target fault diagnosis environment; the processing module is also used to display the target fault diagnosis environment to the user through the AR device.

[0021] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods described above.

[0022] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any of the methods described above is executed.

[0023] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By acquiring the switch cabinet image collected by the AR device, the internal structure of the switch cabinet is identified using the preset three-dimensional model recognition library to obtain the spatial position information of each component, and a virtual fault diagnosis environment for the switch cabinet is constructed accordingly. At the same time, the temperature distribution matrix of each component is obtained, a thermal cloud map is generated, the problem component is identified and superimposed on the virtual fault diagnosis environment to form a target fault diagnosis environment, which is finally displayed to the user through the AR device. This method combines AR technology with infrared thermal imaging to achieve a visual and immersive diagnostic experience for switch cabinet faults. Maintenance personnel can wear AR glasses to see at a glance the internal structure of the switch cabinet and the location, temperature and other information of the faulty components, thereby improving diagnostic efficiency and accuracy.

[0024] 2. By extracting the feature information of the switch cabinet image and comparing it with the standard feature information, the corresponding 3D model is matched, and the internal structure of the switch cabinet is reconstructed in 3D to obtain the spatial position information of each component. This method realizes the rapid and automatic generation of the 3D model of the switch cabinet. The accuracy of the 3D reconstruction is guaranteed by the feature comparison and matching method, so that the 3D model can truly reflect the internal structure of the switch cabinet.

[0025] 3. Based on the spatial position information of each component of the switch cabinet, a three-dimensional virtual model of each component is generated, and the working parameters of the switch cabinet are obtained. The working parameters are associated with the virtual component model to build a virtual fault diagnosis environment for the switch cabinet. This method not only realizes the three-dimensional visualization of the physical structure of the switch cabinet, but also integrates the key parameters that reflect its working status, so that the virtual fault diagnosis environment can comprehensively and dynamically simulate the actual operation of the switch cabinet. Maintenance personnel can monitor the working parameters of each component in real time in the virtual fault diagnosis environment, understand the health status of the equipment, and predict the possible types of faults. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flowchart of an AR-based intelligent switch cabinet fault diagnosis method disclosed in an embodiment of the present application; Figure 2 It is a module schematic diagram of an AR-based intelligent switch cabinet fault diagnosis device disclosed in an embodiment of the present application; Figure 3 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application.

[0027] Explanation of the reference numerals: 201, acquisition module; 202, processing module; 300, electronic device; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0028] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0029] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0030] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0031] This application provides an AR-based intelligent switch cabinet fault diagnosis method, referring to Figure 1 , Figure 1 1 is a flowchart of an AR-based intelligent switch cabinet fault diagnosis method provided in an embodiment of the present application. The method is applied to a server, and the server is used to execute the intelligent switch cabinet fault diagnosis program. The server can be a single server, or a server cluster composed of multiple servers, or a cloud computing service center. The server can communicate with the AR device through a wired or wireless network. The method includes steps S101 to S106, and the above steps are as follows: Step S101: Acquire a switch cabinet image captured by an AR device.

[0032] In step S101, a communication connection is established between the AR device and the server. The communication connection can be a wired connection, such as a connection through Ethernet, USB, etc.; it can also be a wireless connection, such as a connection through WiFi, Bluetooth, 4G / 5G, etc. Secondly, the AR device is equipped with an image acquisition unit, such as a camera, a depth camera, etc., for collecting image data of the switch cabinet. The AR device can start the image acquisition unit according to the user's operating instructions, or according to preset automatic trigger conditions (such as timed shooting, detection of the switch cabinet door opening, etc.), and take pictures or videos of the switch cabinet. The AR device transmits the collected switch cabinet image to the server in real time through the communication connection established above. After receiving the switch cabinet image transmitted by the AR device, the server decompresses and decodes the switch cabinet image to restore the original switch cabinet image.

[0033] For example, on-site maintenance personnel wear AR glasses to conduct routine inspections of a smart switch cabinet. When the maintenance personnel come to the switch cabinet and open the cabinet door, the AR glasses automatically wake up the image acquisition function and use the built-in high-definition camera to capture images of the inside of the switch cabinet from multiple angles at a speed of 30 frames per second. After the shooting is completed, the AR glasses upload the switch cabinet image to the server in real time through the built-in 5G communication module.

[0034] Before step S101, the method further includes: determining whether the door of the switch cabinet is in an open state; if it is determined that the door of the switch cabinet is not in an open state, displaying a prompt message, the prompt message being used to prompt the user to open the door of the switch cabinet.

[0035] Specifically, the server preprocesses the received switch cabinet image, such as image denoising and enhancement. After preprocessing, the server uses a deep learning-based target detection algorithm, such as YOLO and SSD, to locate and identify the switch cabinet door in the switch cabinet image. The server determines the open state of the cabinet door based on the position characteristics and shape characteristics of the cabinet door area. For example, the server matches the cabinet door characteristics in the closed state by analyzing the texture characteristics and color appearance characteristics of the cabinet door area. If the similarity is greater than a preset similarity threshold, the cabinet door is considered to be in a closed state. If the server determines that the cabinet door of the switch cabinet is not in an open state, a prompt message is sent to the AR device to prompt the user to open the cabinet door. The prompt message can be presented in a variety of forms such as text, voice, and graphics. For example, a text reminder of "Please open the switch cabinet door" pops up on the display screen of the AR device, or a voice prompt of "The switch cabinet door is in a closed state, please open the cabinet door to continue the operation" is played.

[0036] Step S102: Based on the switch cabinet image, the internal structure of the switch cabinet is identified using a preset three-dimensional model recognition library to obtain spatial position information of each component of the switch cabinet.

[0037] In step S102, feature information of the switch cabinet image is extracted, and the feature information is compared with standard feature information in a preset three-dimensional model recognition library to obtain a comparison result; based on the comparison result, a three-dimensional model corresponding to the switch cabinet image is matched; the internal structure of the switch cabinet is reconstructed based on the three-dimensional model to obtain spatial position information of each component of the switch cabinet.

[0038] Specifically, after receiving the switch cabinet image captured by the AR device, the server first preprocesses the switch cabinet image, including denoising, enhancement and correction. Then, the server uses a deep learning model (such as a convolutional neural network) to extract the feature information of the switch cabinet from the switch cabinet image, including the geometric features of the cabinet appearance (such as size and contour), component features (such as the position and shape of circuit breakers, transformers, and terminals), surface features (such as texture and color), and structural features (such as the relative position relationship between components). After extracting the feature information, the server converts the feature information into a standardized feature vector and performs a multi-dimensional comparison with the standard feature information stored in the preset three-dimensional model recognition library. The comparison process uses a feature matching algorithm, such as SIFT or SURF algorithm, to calculate the similarity score between the image feature and each standard feature. Based on the comprehensive scoring results, the server selects the three-dimensional model with the highest similarity and exceeding the preset similarity threshold (such as 90%) as the best matching result. After obtaining the matching three-dimensional model, the server establishes a unified three-dimensional coordinate system and maps the standard component position information in the three-dimensional model to the actual coordinate system. The server generates a complete three-dimensional structural model including the spatial coordinates, posture parameters, size information and interrelationships of all components.

[0039] Assume that the server receives a front image of a certain type of switch cabinet and first identifies its feature information: a standard cabinet with a height of 2000 mm and a width of 800 mm, a black handle on the right, four instrument display windows on the top, and three indicator lights in the middle. The server compares this feature information with the preset 3D model recognition library and matches a 3D model with a similarity of 95%. The 3D model contains complete internal structure information, and the server builds a 3D structural model of the switch cabinet based on this.

[0040] Step S103: constructing a virtual fault diagnosis environment of the switch cabinet according to the spatial position information of each component.

[0041] In step S103, a three-dimensional virtual model of each component is generated based on the spatial position information of each component; the operating parameters of the switch cabinet are obtained, and the operating parameters are associated with the three-dimensional virtual models of each component to obtain a virtual fault diagnosis environment of the switch cabinet; wherein the operating parameters include voltage, current and power.

[0042] Specifically, the server generates a 3D virtual model of each component based on the spatial position information of each component of the switch cabinet. The server matches the corresponding standard 3D model from the preset 3D model recognition library according to the type and model of the component. Then, the server adjusts and optimizes the size, position, orientation and other parameters of the standard 3D model according to the spatial position information of the actual switch cabinet to obtain a 3D virtual model that matches the actual component.

[0043] Secondly, the server obtains the real-time working parameters of the switch cabinet, including the voltage, current, power and other electrical parameters of each component. These parameters can be collected by the intelligent sensors (such as mutual inductors, voltage transformers, etc.) built into the switch cabinet, or by external measuring instruments (such as power quality analyzers, infrared thermal imagers, etc.). The server reads real-time data from sensors or instruments through wired or wireless communication, and performs necessary pre-processing operations such as data conversion, calibration, filtering, etc. to obtain accurate working parameters.

[0044] The server then associates and maps the obtained switch cabinet working parameters with the generated 3D virtual models of the components. The server assigns the corresponding working parameters to the corresponding virtual models according to the type and location of each component. Through this parameter mapping, the 3D virtual model of each component can reflect the actual working status of the component.

[0045] Finally, the server integrates the 3D virtual model and working parameters of the components to build a complete virtual fault diagnosis environment for the switch cabinet, realizing the visual display and interactive operation of the switch cabinet. Maintenance personnel can connect to the server through AR devices to view the 3D virtual model of the switch cabinet and monitor the working parameters of each component in real time.

[0046] Step S104: obtaining the temperature distribution matrix of each component, and generating a thermal cloud map of each component according to the temperature distribution matrix.

[0047] In step S104, the temperature distribution matrix of each component is obtained, which specifically includes: controlling the infrared thermal imager to collect infrared thermal images of each component; performing image segmentation processing on the infrared thermal images to obtain the temperature distribution area of ​​each component; extracting the temperature value of each temperature distribution area to generate the temperature distribution matrix of each component.

[0048] Generate a thermal cloud map of each component according to the temperature distribution matrix, specifically including: normalizing the temperature distribution matrix to obtain a normalized temperature distribution matrix; constructing a thermal cloud map color scale, which includes color identifiers corresponding to different temperature intervals; traversing each element in the normalized temperature distribution matrix to obtain a normalized temperature value of each element; determining a target color identifier in the thermal cloud map color scale corresponding to the normalized temperature value according to the normalized temperature value; in a virtual fault diagnosis environment, superimposing and rendering the target color identifier according to the position of each element in the normalized temperature distribution matrix to obtain a thermal cloud map of each component.

[0049] Specifically, the server controls the infrared thermal imager to collect infrared thermal images of various components of the switch cabinet. The infrared thermal imager is a measuring device that uses infrared sensors to detect the temperature distribution on the surface of an object, and can generate high-resolution, high-sensitivity thermal images. The server establishes a communication connection with the infrared thermal imager via wired or wireless means, sends acquisition instructions, and receives infrared thermal image data returned by the infrared thermal imager. Secondly, the server performs image segmentation processing on the acquired infrared thermal image to extract the temperature distribution area of ​​each component. The server uses a segmentation algorithm to segment the infrared thermal image into different temperature distribution areas such as the circuit breaker area, busbar area, and cable joint area based on the geometric shape, spatial position and other characteristics of the switch cabinet components.

[0050] Then, the server extracts the temperature value of each segmented temperature distribution area and generates the temperature distribution matrix of the corresponding component. The temperature distribution matrix is ​​a two-dimensional array, in which each element represents the temperature value of the corresponding position in the temperature distribution area. The server analyzes the gray value or pseudo-color value of each pixel in the infrared thermal image, combines the temperature calibration parameters of the infrared thermal imager, calculates the actual temperature value of each pixel, and fills it into the corresponding position of the temperature distribution matrix. In this way, the temperature distribution of each component is quantified into a temperature distribution matrix.

[0051] Next, the server generates a thermal cloud map for each component based on the temperature distribution matrix. A thermal cloud map is a visual method that displays temperature distribution in an intuitive and vivid way, and uses color changes to indicate the high and low temperature values. The server first normalizes the temperature distribution matrix, maps the temperature values ​​to the interval [0, 1], and obtains a normalized temperature distribution matrix. Then, the server constructs a thermal cloud map color scale to associate different temperature intervals with specific color identifiers. Thermal cloud map color scales include red-yellow-blue scales, rainbow scales, etc. The server selects based on the dynamic range of temperature distribution and display requirements.

[0052] Finally, the server traverses each element in the normalized temperature distribution matrix, searches for the corresponding color identifier in the thermal cloud color code according to the normalized temperature value of the element, and overlays and renders the color identifier to the corresponding position of the 3D virtual model of the component in the virtual fault diagnosis environment. In this way, a thermal cloud map of each component is generated, which intuitively shows the temperature distribution of the component.

[0053] Step S105: Determine the problem component according to the thermal cloud map, and superimpose the problem component into the virtual fault diagnosis environment to obtain the target fault diagnosis environment.

[0054] In step S105, the average temperature value of each component is calculated according to the thermal cloud map; the average temperature value of each component is compared with the preset temperature threshold to obtain the problem component, which is a component with an average temperature value greater than or equal to the preset temperature threshold; in the virtual fault diagnosis environment, the target position of the problem component is identified, and the thermal cloud map corresponding to the problem component is superimposed on the target position to obtain the target fault diagnosis environment.

[0055] Specifically, the server calculates the average temperature value of each component based on the thermal cloud map of each component. The average temperature value reflects the overall temperature level of the component and is an important indicator for evaluating the temperature status of the component. The server obtains the average temperature value of each component by arithmetically averaging the temperature values ​​of all pixels in the thermal cloud map. Secondly, the server compares the calculated average temperature value of each component with the preset temperature threshold to screen out the problematic components. The preset temperature threshold is the upper limit of the safe temperature specified by experts or standards based on factors such as the material properties, structural characteristics, and working conditions of the switch cabinet components, and this application does not limit this. Different types of components can be set with different preset temperature thresholds to adapt to their temperature characteristic differences. The server determines whether the component has a temperature anomaly by comparing the size relationship between the average temperature value of the component and the corresponding preset temperature threshold. If the average temperature value of the component is greater than or equal to the preset temperature threshold, the component is marked as a problematic component, indicating that its temperature exceeds the safe range and there may be a hidden danger of failure.

[0056] Finally, the server overlays and renders the thermal cloud map corresponding to the problem component onto the target position of the virtual fault diagnosis environment to obtain the target fault diagnosis environment. The target fault diagnosis environment is an enhanced virtual scene that overlays the thermal cloud map of the problem component on the original virtual fault diagnosis environment. The server uses texture mapping of the graphics rendering pipeline (such as OpenGL, DirectX, etc.) to overlay the thermal cloud map of the problem component as a texture onto the surface of the three-dimensional virtual model of the corresponding component. The color information of the thermal cloud map is combined with the geometric information of the component model to intuitively display the temperature distribution state of the problem component. In the target fault diagnosis environment, the problem component area will be prominently marked with high temperature colors (such as red and orange) in the thermal cloud map to attract the attention of maintenance personnel.

[0057] For example, by analyzing the thermal cloud map of each component of the switch cabinet, the server calculated that the average temperature values ​​of the circuit breaker, transformer, cable connector and other components were 65 degrees Celsius, 45 degrees Celsius, and 80 degrees Celsius respectively. After comparing with the preset temperature threshold (such as 60 degrees Celsius), the server found that the average temperature values ​​of the circuit breaker and cable connector exceeded the preset temperature threshold and determined them to be problematic components.

[0058] Step S106: displaying the target fault diagnosis environment to the user through the AR device.

[0059] In step S106, the server establishes a communication connection with the AR device to achieve two-way transmission of data and instructions. An AR device is a wearable device that overlays virtual information onto a real environment, and usually includes components such as a head-mounted display and a handheld controller. The server pairs with the AR device through wired or wireless means such as WiFi, Bluetooth, and USB to establish a stable, high-speed data channel. The server acts as the master terminal and the AR device acts as the slave terminal, and two-way data exchange is achieved through a specific communication protocol.

[0060] Secondly, the server transmits the data model of the target fault diagnosis environment to the AR device and controls the AR device to render the scene. The target fault diagnosis environment is a complex data structure that contains information such as the three-dimensional model of the switch cabinet and the thermal cloud map of the problem component. The server efficiently sends the data model of the target fault diagnosis environment to the AR device through technologies such as data compression and fragmented transmission. After receiving the data model, the AR device uses its own graphics rendering engine to overlay the virtual scene onto the real environment.

[0061] Then, by wearing an AR device, the user can immersively explore and interact with the target fault diagnosis environment. The user can browse and operate the virtual scene through natural interaction methods such as head movement, gesture control, and voice commands. For example, the user can call up the working parameter panel of a component by looking at it; the user can use gestures to draw a circle in the virtual space and select the component within the circle. The AR device transmits the user's operation instructions to the server, which performs corresponding data processing and scene updates, and then feeds back the results to the AR device to present to the user.

[0062] Finally, the user uses the AR device to analyze and make decisions on the target fault diagnosis environment and formulate a fault repair plan. In the immersive virtual scene, the user can intuitively observe the overall structure and layout of the switch cabinet and find the specific location of the problem component in the switch cabinet. Through the color identification of the thermal cloud map, the user can quickly determine the temperature abnormality of the problem component.

[0063] Reference Figure 2 The present application also provides an AR-based intelligent switch cabinet fault diagnosis device, which is a server, and the server includes an acquisition module 201 and a processing module 202, wherein: the acquisition module 201 is used to acquire the switch cabinet image collected by the AR device; the processing module 202 is used to identify the internal structure of the switch cabinet based on the switch cabinet image using a preset three-dimensional model recognition library to obtain the spatial position information of each component of the switch cabinet; the processing module 202 is also used to construct a virtual fault diagnosis environment of the switch cabinet according to the spatial position information of each component; the acquisition module 201 is also used to acquire the temperature distribution matrix of each component, and generate a thermal cloud map of each component according to the temperature distribution matrix; the processing module 202 is also used to determine the problem component according to the thermal cloud map, and superimpose the problem component on the virtual fault diagnosis environment to obtain the target fault diagnosis environment; the processing module 202 is also used to display the target fault diagnosis environment to the user through the AR device.

[0064] In a possible implementation, the processing module 202 identifies the internal structure of the switch cabinet based on the switch cabinet image by using a preset three-dimensional model recognition library to obtain spatial position information of each component of the switch cabinet, specifically including: the processing module 202 extracts feature information of the switch cabinet image, and compares the feature information with standard feature information in the preset three-dimensional model recognition library to obtain a comparison result; the processing module 202 matches the three-dimensional model corresponding to the switch cabinet image according to the comparison result; and reconstructs the internal structure of the switch cabinet according to the three-dimensional model to obtain spatial position information of each component of the switch cabinet.

[0065] In a possible implementation, the processing module 202 constructs a virtual fault diagnosis environment for the switch cabinet according to the spatial position information of each component, specifically including: the processing module 202 generates a three-dimensional virtual model of each component based on the spatial position information of each component; the acquisition module 201 acquires the working parameters of the switch cabinet, and associates the working parameters with the three-dimensional virtual models of each component to obtain a virtual fault diagnosis environment for the switch cabinet; wherein the working parameters include voltage, current and power.

[0066] In a possible implementation, the acquisition module 201 acquires the temperature distribution matrix of each component, specifically including: the processing module 202 controls the infrared thermal imager to collect infrared thermal images of each component; performs image segmentation processing on the infrared thermal image to obtain the temperature distribution area of ​​each component; the processing module 202 extracts the temperature value of each temperature distribution area to generate the temperature distribution matrix of each component.

[0067] In a possible implementation, the processing module 202 generates a thermal cloud map of each component according to the temperature distribution matrix, specifically including: the processing module 202 normalizes the temperature distribution matrix to obtain a normalized temperature distribution matrix; the processing module 202 constructs a thermal cloud map color code, and the thermal cloud map color code includes color identifiers corresponding to different temperature intervals; the acquisition module 201 traverses each element in the normalized temperature distribution matrix to obtain a normalized temperature value of each element; the processing module 202 determines the target color identifier in the thermal cloud map color code corresponding to the normalized temperature value according to the normalized temperature value; the processing module 202, in a virtual fault diagnosis environment, overlays and renders the target color identifier according to the position of each element in the normalized temperature distribution matrix to obtain a thermal cloud map of each component.

[0068] In one possible implementation, the processing module 202 determines the problem component based on the thermal cloud map, and superimposes the problem component to the virtual fault diagnosis environment to obtain a target fault diagnosis environment, specifically including: the processing module 202 calculates the average temperature value of each component based on the thermal cloud map; the processing module 202 compares the average temperature value of each component with a preset temperature threshold to obtain the problem component, and the problem component is a component with an average temperature value greater than or equal to the preset temperature threshold; the processing module 202 identifies the target position of the problem component in the virtual fault diagnosis environment, and superimposes the thermal cloud map corresponding to the problem component to the target position to obtain the target fault diagnosis environment.

[0069] In a possible implementation, after the acquisition module 201 acquires the switch cabinet image captured by the AR device, the method further includes: the processing module 202 determines whether the cabinet door of the switch cabinet is in an open state based on the switch cabinet image; if the processing module 202 determines that the cabinet door of the switch cabinet is not in an open state, a prompt message is displayed, and the prompt message is used to prompt the user to open the cabinet door of the switch cabinet.

[0070] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0071] The present application also provides an electronic device. Figure 3 , Figure 3 The electronic device 300 may include: at least one processor 301 , at least one network interface 304 , a user interface 303 , a memory 305 , and at least one communication bus 302 .

[0072] The communication bus 302 is used to realize the connection and communication between these components.

[0073] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0074] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0075] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0076] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may optionally also be at least one storage device located away from the aforementioned processor 301. Refer to Figure 3 , the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program of an AR-based intelligent switch cabinet fault diagnosis method.

[0077] exist Figure 3In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 301 can be used to call the application program storing an AR-based intelligent switch cabinet fault diagnosis method in the memory 305, and when executed by one or more processors 301, the electronic device 300 executes one or more of the methods described in the above embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0078] The present application also provides a computer-readable storage medium, which stores instructions. When executed by one or more processors 301, the electronic device 300 executes one or more of the methods described in the above embodiments.

[0079] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0080] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0081] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0082] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0083] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0084] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0085] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A fault diagnosis method for intelligent switch cabinet based on AR, characterized in that: The method comprises: Obtain the switch cabinet image captured by the AR device; Based on the switch cabinet image, the internal structure of the switch cabinet is identified using a preset three-dimensional model recognition library to obtain spatial position information of each component of the switch cabinet; Constructing a virtual fault diagnosis environment of the switch cabinet according to the spatial position information of each of the components; Acquire a temperature distribution matrix of each of the components, and generate a thermal cloud map of each of the components according to the temperature distribution matrix; Determine the problem component according to the thermal cloud map, and superimpose the problem component to the virtual fault diagnosis environment to obtain a target fault diagnosis environment; The target fault diagnosis environment is presented to the user through the AR device.

2. The method according to claim 1, characterized in that Based on the switch cabinet image, the internal structure of the switch cabinet is identified by using a preset three-dimensional model recognition library to obtain the spatial position information of each component of the switch cabinet, specifically including: Extracting feature information of the switch cabinet image, and comparing the feature information with standard feature information in the preset three-dimensional model recognition library to obtain a comparison result; According to the comparison result, a three-dimensional model corresponding to the switch cabinet image is matched; The internal structure of the switch cabinet is reconstructed according to the three-dimensional model to obtain the spatial position information of each component of the switch cabinet.

3. The method according to claim 1, characterized in that The step of constructing a virtual fault diagnosis environment of the switch cabinet according to the spatial position information of each of the components specifically includes: Based on the spatial position information of each of the components, generating a three-dimensional virtual model of each of the components; The operating parameters of the switch cabinet are acquired, and the operating parameters are associated with the three-dimensional virtual models of each of the components to obtain a virtual fault diagnosis environment of the switch cabinet; wherein the operating parameters include voltage, current and power.

4. The method according to claim 1, characterized in that: The obtaining of the temperature distribution matrix of each of the components specifically includes: Controlling the infrared thermal imager to collect infrared thermal images of each of the components; Performing image segmentation processing on the infrared thermal image to obtain the temperature distribution area of ​​each of the components; The temperature values ​​of each of the temperature distribution areas are extracted to generate a temperature distribution matrix for each of the components.

5. The method according to claim 4, characterized in that Generating a thermal cloud map of each of the components according to the temperature distribution matrix specifically includes: Normalizing the temperature distribution matrix to obtain a normalized temperature distribution matrix; Constructing a thermal cloud map color scale, wherein the thermal cloud map color scale includes color identifiers corresponding to different temperature intervals; Traversing each element in the normalized temperature distribution matrix to obtain a normalized temperature value of each element; According to the normalized temperature value, determining a target color identifier corresponding to the normalized temperature value in the thermal cloud map color scale; In the virtual fault diagnosis environment, the target color identification is superimposed and rendered according to the position of each of the elements in the normalized temperature distribution matrix to obtain a thermal cloud map of each of the components.

6. The method according to claim 1, characterized in that The determining of the problem component according to the thermal cloud map and superimposing the problem component to the virtual fault diagnosis environment to obtain a target fault diagnosis environment specifically includes: Calculating the average temperature of each component according to the thermal cloud map; Compare the average temperature value of each of the components with a preset temperature threshold to obtain a problematic component, wherein the problematic component is a component whose average temperature value is greater than or equal to the preset temperature threshold; In the virtual fault diagnosis environment, the target position of the problem component is identified, and the thermal cloud map corresponding to the problem component is superimposed on the target position to obtain the target fault diagnosis environment.

7. The method according to claim 1, characterized in that After acquiring the switch cabinet image acquired by the AR device, the method further includes: According to the switch cabinet image, determining whether a cabinet door of the switch cabinet is in an open state; If it is determined that the cabinet door of the switch cabinet is not in the open state, a prompt message is displayed, where the prompt message is used to prompt the user to open the cabinet door of the switch cabinet.

8. An AR-based intelligent switch cabinet fault diagnosis device, characterized in that: The device comprises an acquisition module (201) and a processing module (202), wherein: The acquisition module (201) is used to acquire the switch cabinet image collected by the AR device; The processing module (202) is used to identify the internal structure of the switch cabinet based on the switch cabinet image using a preset three-dimensional model recognition library to obtain spatial position information of each component of the switch cabinet; The processing module (202) is further used to construct a virtual fault diagnosis environment of the switch cabinet according to the spatial position information of each of the components; The acquisition module (201) is further used to acquire a temperature distribution matrix of each of the components, and generate a thermal cloud map of each of the components according to the temperature distribution matrix; The processing module (202) is further used to determine the problem component according to the thermal cloud map, and superimpose the problem component on the virtual fault diagnosis environment to obtain a target fault diagnosis environment; The processing module (202) is further configured to display the target fault diagnosis environment to the user via the AR device.

9. An electronic device, characterized in that: The electronic device (300) comprises a processor (301), a memory (305), a user interface (303) and a network interface (304), wherein the memory (305) is used to store instructions, the user interface (303) and the network interface (304) are used to communicate with other devices, and the processor (301) is used to execute the instructions stored in the memory (305) so that the electronic device (300) executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Images