Device running state recognition method and device, imager, and storage medium

Abstract

The application discloses a device running state recognition method and device, an imager and a storage medium, and belongs to the technical field of industrial detection. The application realizes automatic recognition of the running state of the measured device based on signals collected in a target scene and reflecting the running state of the measured device. Since manual running state recognition of the measured device is not required, not only manpower cost is saved, but also efficiency is higher, abnormal conditions of the measured device can be found and handled in time, and the reliability and stability of device running are improved. In addition, in addition to outputting the running state recognition result, the scheme also performs visual display based on the generated image. The generated image and the running state recognition result are used to represent the current running state of the measured device from different angles. This visual presentation mode enables an operator to observe and visually check abnormal features such as sound, heat and discharge of the device, and helps the operator to efficiently master the running state of the device.

Description

Technical Field

[0001] This application relates to the field of industrial testing technology, and in particular to a method, apparatus, imager, and storage medium for identifying the operating status of equipment. Background Technology

[0002] In industrial production, equipment malfunctions can not only affect production efficiency but also lead to equipment failures and even safety accidents. Therefore, identifying the operating status of equipment is a crucial step in industrial production.

[0003] Currently, most equipment operation status is identified through manual inspection. However, manual inspection is not only inefficient, but also prone to missing some anomalies in a timely manner, which can seriously affect the reliability and stability of equipment operation. Summary of the Invention

[0004] This application provides a method, apparatus, imager, and storage medium for identifying the operating status of a device. The technical solution is shown below.

[0005] On the one hand, a method for identifying the operating status of a device is provided, the method comprising: Acquire acoustic signals collected in the target scene; Using the device type of the device under test in the target scene as a constraint, at least one image is generated based on the acoustic signal; Based on the acoustic signal, the operating status of the device under test is identified to obtain an operating status identification result; wherein, the at least one image and the operating status identification result are used to characterize the current operating status of the device under test from different perspectives; The operating status identification result is output and visualized based on the at least one image.

[0006] In some embodiments, the method further includes: Acquire the thermal imaging signal radiated by the device under test; The generation of at least one image based on the acoustic signal includes: Using the device type as a constraint, the at least one image is generated based on one or more of the acoustic signal or the thermal imaging signal.

[0007] In other embodiments, when the device under test is an electrical device, generating the at least one image based on one or more of the acoustic signal or the thermal imaging signal includes: Based on the acoustic signal, a PRPD (Phase Resolved Partial Discharge) map is generated in real time. The PRPD map is used to describe the partial discharge characteristics of the device under test through phase distribution. A thermal imaging image is generated based on the thermal imaging signal, and the thermal imaging image is used to describe the temperature distribution on the surface of the device under test in a visual manner. A sound pressure distribution image is generated based on the acoustic signal. The sound pressure distribution image is used to describe the sound pressure intensity at each spatial location in the target scene. Based on the mapping relationship between sound pressure intensity and color, the sound pressure distribution image is visualized and drawn to obtain an acoustic imaging map covering the target scene. The thermal features of the thermal imaging image and the acoustic features of the acoustic imaging image are extracted; the thermal features and the acoustic features are fused to obtain multimodal features; the multimodal features are visualized to obtain an acoustic-thermal anomaly correlation map; wherein, the acoustic-thermal anomaly correlation map is used to describe the degree of correlation between acoustic anomalies and thermal anomalies at various locations of the tested device; the acoustic anomaly refers to the abnormal operation of the device identified by the acoustic signal; the thermal anomaly refers to the abnormal operation of the device identified by the thermal imaging signal.

[0008] In other embodiments, the PRPD map generation process includes: Feature extraction is performed on the acoustic signal to obtain an effective pulse signal; the PRPD map is generated based on the effective pulse signal; wherein, the feature extraction includes at least one of the following: Temporal feature extraction is performed on the preprocessed acoustic signal; Frequency domain features are extracted from the preprocessed acoustic signal; Time-frequency domain features are extracted from the preprocessed acoustic signal.

[0009] In other embodiments, generating the PRPD map based on the effective pulse signal includes: obtaining the amplitude and trigger timestamp of each pulse included in the effective pulse signal; Acquire phase data of the power frequency voltage signal that is synchronously acquired with the acoustic signal; the phase data includes the phase of the power frequency voltage signal at each moment; Using the zero-crossing point of the power frequency voltage signal as the zero phase, a mapping relationship between time and phase is established; Based on the trigger timestamp of each pulse and the mapping relationship, the phase of each pulse is determined, and a two-dimensional data pair of each pulse is obtained; wherein, for any pulse, the two-dimensional data pair of the pulse includes the amplitude and phase of the pulse; The PRPD map is generated based on the two-dimensional data pairs of each pulse.

[0010] In other embodiments, generating the PRPD map based on two-dimensional data pairs for each pulse includes: The phase is divided into multiple phase intervals according to a preset step size; The pulse amplitude is divided into multiple amplitude intervals according to linear or logarithmic intervals; Based on the two-dimensional data pairs of each pulse, the number of pulses contained in each phase-amplitude unit is counted; With phase as the horizontal axis and pulse amplitude as the vertical axis, each phase-amplitude unit is visualized based on the statistical number of pulses to obtain the PRPD spectrum; In the PRPD spectrum, the multiple phase intervals are uniformly distributed on the horizontal axis, and the multiple amplitude intervals are uniformly distributed on the vertical axis.

[0011] In other embodiments, the visualization based on the at least one image includes: Obtain a real-world image of the target scene, and use the real-world image as a base image to overlay and display the acoustic imaging image; or, Simultaneously display the PRPD map and the thermal image; or, Simultaneously display the acoustic image and the thermal image; or, Display a fused image of the real-world image, the acoustic image, and the thermal image; use the fused image as a base image and overlay the PRPD map onto the fused image; or, The correlation diagram of the aforementioned acoustic and thermal anomalies is shown.

[0012] In other embodiments, the method further includes: A real-world image of the target scene is acquired, and target detection is performed on the real-world image to obtain a target detection result; the target detection result includes the device type and location coordinates of the device under test; Thermal anomaly detection is performed based on the thermal image to obtain thermal anomaly detection results; the thermal anomaly detection results include the contour coordinates of the detected thermal anomaly region. Based on the target detection results and the thermal anomaly detection results, the candidate detection area of ​​the device under test is determined; The candidate detection area is marked on the real-world image and the thermal image, and a prompt message is output; the prompt message is used to prompt the imager to be aligned with the candidate detection area.

[0013] In other embodiments, the method further includes: Based on coordinate transformation rules, the pixel coordinates of the candidate detection region in the image space are converted into the acoustic coordinates or beam pointing parameters of the acoustic signal acquisition module of the imager in the detection space.

[0014] In other embodiments, the step of identifying the operating status of the device under test based on the acoustic signal includes: Determine a set of acoustic features used to identify the operating status of the device under test; obtain an anomaly threshold for each acoustic feature in the set; obtain the feature value of each acoustic feature; if the obtained feature value satisfies the threshold conditions corresponding to a preset number of acoustic features, determine that the device under test has an operating anomaly; or, Invoke a machine learning model that matches the device type and perform the step of identifying the operating status of the device under test based on the acoustic signal.

[0015] In other embodiments, the method further includes: A structured detection record is generated; the structured detection record is obtained by encapsulating at least one of the following: first scene information of the target scene, candidate detection area of ​​the device under test, thermal anomaly data reflecting the surface temperature of the device under test, acoustic features of the acoustic signal, parameter information of the PRPD spectrum, or the operating status identification result; Based on the structured detection records and industry knowledge, a prompt text is constructed; the prompt text includes at least one of the following: second scene information of the target scene, output constraint information, or abnormal evaluation information of the tested device; the second scene information has a greater amount of information than the first scene information; Using the prompt text as input to the large language model, the target text is generated by the large language model; wherein, the target text includes at least one of the following: an overview of the current operation status identification, explanatory information of the operation status identification results, explanatory information on the risk level, and processing and retesting suggestions.

[0016] In other embodiments, the step of using the prompt text as input to a large language model and generating target text through the large language model includes: Using the prompt text as input to the large language model, the large language model generates target text that conforms to the text output framework. The text output framework is matched with the target scene.

[0017] In other embodiments, the method further includes: Get user questions; The user's question and the structured detection record are input into the large language model, and the response content from the large language model is obtained. Output the reply content.

[0018] In other embodiments, a report generation button is displayed on the display screen; the method further includes: outputting a device status report in response to a triggering operation of the report generation button; The equipment status report includes at least one of the following: The at least one image; Sound source location coordinates, which are used to indicate the position of the device under test in the target scene; Temperature trend curve, which is used to describe the change trend of the surface temperature of the device under test over time; The structured detection record; The target text.

[0019] On the other hand, a device for identifying the operating status of a device is provided, the device comprising: The first acquisition module is configured to acquire acoustic signals collected in the target scene; The first generation module is configured to generate at least one image based on the acoustic signal, using the device type of the device under test in the target scene as a constraint. The identification module is configured to identify the operating status of the device under test based on the acoustic signal, and obtain an operating status identification result; wherein, the at least one image and the operating status identification result are used to characterize the current operating status of the device under test from different perspectives; The output module is configured to output the running status recognition result and visualize it based on the at least one image.

[0020] In some embodiments, the first acquisition module is further configured to acquire the thermal imaging signal radiated by the device under test; The first generation module is configured to generate the at least one image based on one or more of the acoustic signal or the thermal imaging signal, using the device type as a constraint.

[0021] In other embodiments, when the device under test is an electrical device, the first generation module is configured to: Based on the acoustic signal, a PRPD spectrum is generated in real time. The PRPD spectrum is used to describe the partial discharge characteristics of the device under test through phase distribution. A thermal imaging image is generated based on the thermal imaging signal, and the thermal imaging image is used to describe the temperature distribution on the surface of the device under test in a visual manner. A sound pressure distribution image is generated based on the acoustic signal. The sound pressure distribution image is used to describe the sound pressure intensity at each spatial location in the target scene. Based on the mapping relationship between sound pressure intensity and color, the sound pressure distribution image is visualized and drawn to obtain an acoustic imaging map covering the target scene. The thermal features of the thermal imaging image and the acoustic features of the acoustic imaging image are extracted; the thermal features and the acoustic features are fused to obtain multimodal features; the multimodal features are visualized to obtain an acoustic-thermal anomaly correlation map; wherein, the acoustic-thermal anomaly correlation map is used to describe the degree of correlation between acoustic anomalies and thermal anomalies at various locations of the tested device; the acoustic anomaly refers to the abnormal operation of the device identified by the acoustic signal; the thermal anomaly refers to the abnormal operation of the device identified by the thermal imaging signal.

[0022] In other embodiments, the first generation module is configured to: Feature extraction is performed on the acoustic signal to obtain an effective pulse signal; the PRPD map is generated based on the effective pulse signal; wherein, the feature extraction includes at least one of the following: Temporal feature extraction is performed on the preprocessed acoustic signal; Frequency domain features are extracted from the preprocessed acoustic signal; Time-frequency domain features are extracted from the preprocessed acoustic signal.

[0023] In other embodiments, the first generation module is configured to: Obtain the amplitude and trigger timestamp of each pulse included in the valid pulse signal; Acquire phase data of the power frequency voltage signal that is synchronously acquired with the acoustic signal; the phase data includes the phase of the power frequency voltage signal at each moment; Using the zero-crossing point of the power frequency voltage signal as the zero phase, a mapping relationship between time and phase is established; Based on the trigger timestamp of each pulse and the mapping relationship, the phase of each pulse is determined, and a two-dimensional data pair of each pulse is obtained; wherein, for any pulse, the two-dimensional data pair of the pulse includes the amplitude and phase of the pulse; The PRPD map is generated based on the two-dimensional data pairs of each pulse.

[0024] In other embodiments, the first generation module is configured to: The phase is divided into multiple phase intervals according to a preset step size; The pulse amplitude is divided into multiple amplitude intervals according to linear or logarithmic intervals; Based on the two-dimensional data pairs of each pulse, the number of pulses contained in each phase-amplitude unit is counted; With phase as the horizontal axis and pulse amplitude as the vertical axis, each phase-amplitude unit is visualized based on the statistical number of pulses to obtain the PRPD spectrum; In the PRPD spectrum, the multiple phase intervals are uniformly distributed on the horizontal axis, and the multiple amplitude intervals are uniformly distributed on the vertical axis.

[0025] In other embodiments, the output module is configured as follows: Obtain a real-world image of the target scene, and use the real-world image as a base image to overlay and display the acoustic imaging image; or, Simultaneously display the PRPD map and the thermal image; or, Simultaneously display the acoustic image and the thermal image; or, Display a fused image of the real-world image, the acoustic image, and the thermal image; use the fused image as a base image and overlay the PRPD map onto the fused image; or, The correlation diagram of the aforementioned acoustic and thermal anomalies is shown.

[0026] In other embodiments, the device further includes: The first acquisition module is further configured to acquire a real-world image of the target scene; The processing module is configured to perform target detection on the real-scene image and obtain target detection results; the target detection results include the device type and location coordinates of the device under test; The processing module is further configured to perform thermal anomaly detection based on the thermal image to obtain thermal anomaly detection results; the thermal anomaly detection results include the contour coordinates of the detected thermal anomaly region; The processing module is further configured to determine the candidate detection area of ​​the device under test based on the target detection result and the thermal anomaly detection result; The output module is further configured to mark the candidate detection area on the real-world image and the thermal image, and output prompt information; the prompt information is used to prompt the imager to be aligned with the candidate detection area.

[0027] In other embodiments, the processing module is further configured to: Based on coordinate transformation rules, the pixel coordinates of the candidate detection region in the image space are converted into the acoustic coordinates or beam pointing parameters of the acoustic signal acquisition module of the imager in the detection space.

[0028] In other embodiments, the identification module is configured as follows: Determine a set of acoustic features used to identify the operating status of the device under test; obtain an anomaly threshold for each acoustic feature in the set; obtain the feature value of each acoustic feature; if the obtained feature value satisfies the threshold conditions corresponding to a preset number of acoustic features, determine that the device under test has an operating anomaly; or, Invoke a machine learning model that matches the device type and perform the step of identifying the operating status of the device under test based on the acoustic signal.

[0029] In other embodiments, the device further includes: The second generation module is configured to generate structured detection records; the structured detection records are obtained by encapsulating at least one of the following: first scene information of the target scene, candidate detection area of ​​the device under test, thermal anomaly data reflecting the surface temperature of the device under test, acoustic features of the acoustic signal, parameter information of the PRPD spectrum, or the operating status identification result. The construction module is configured to construct prompt text based on the structured detection records and industry knowledge; the prompt text includes at least one of the following: second scene information of the target scene, output constraint information, or abnormal evaluation information of the tested device; the second scene information has a greater amount of information than the first scene information; The third generation module is configured to use the prompt text as input to the large language model and generate target text through the large language model; wherein the target text includes at least one of the following: an overview of the current operation status identification, explanatory information of the operation status identification results, explanatory information on the risk level, and processing and retesting suggestions.

[0030] In other embodiments, the third generation module is configured as follows: Using the prompt text as input to the large language model, the large language model generates target text that conforms to the text output framework. The text output framework is matched with the target scene.

[0031] In other embodiments, the device further includes: The second acquisition module is configured to acquire user questions; The input module is configured to input the user's question and the structured detection record into the large language model. The third acquisition module is configured to acquire the response content fed back by the large language model; The output module is also configured to output the response content.

[0032] In other embodiments, a report generation button is displayed on the screen; the output module is further configured to output a device status report in response to a triggering operation of the report generation button; wherein the device status report includes at least one of the following: The at least one image; Sound source location coordinates, which are used to indicate the position of the device under test in the target scene; Temperature trend curve, which is used to describe the change trend of the surface temperature of the device under test over time; The structured detection record; The target text.

[0033] On the other hand, an imager is provided, the imager comprising: an acoustic signal acquisition module, a thermal imaging module, and a main body; The acoustic signal acquisition module is fixedly connected to the main body; the thermal imaging module is snapped into the main body; the main body is electrically connected to both the acoustic signal acquisition module and the thermal imaging module. The main body includes a device operation status identification component, which is configured to execute the above-described device operation status identification method based on one or more of the acoustic signals acquired by the acoustic signal acquisition module or the thermal imaging signals acquired by the thermal imaging module.

[0034] On the other hand, a computer-readable storage medium is provided, wherein computer program code is stored in the storage medium and is loaded and executed by a processor to implement the above-described device operating status identification method.

[0035] On the other hand, a computer program product is provided, the computer program product including computer program code stored in a computer-readable storage medium, the processor of the device operation status recognition component reading the computer program code from the computer-readable storage medium, the processor executing the computer program code, causing the imager to perform the above-described device operation status recognition method.

[0036] This application embodiment achieves automatic identification of the operating status of the device under test (DUT) based on signals collected in the target scene that reflect the DUT's operating status. Since no manual identification of the DUT's operating status is required, compared with manual inspection methods, it not only saves labor costs but is also more efficient. Abnormalities of the DUT can be detected and handled in a timely manner, significantly improving the reliability and stability of the equipment's operation.

[0037] In addition to outputting the operational status identification results, this solution also provides a visual display based on at least one generated image. The generated image and the operational status identification results are used to characterize the current operational status of the device under test from different perspectives. This visualization method allows operators to intuitively view the acoustic, thermal, and discharge anomaly characteristics of the equipment, helping them efficiently grasp the equipment's operational status. Furthermore, this solution, with its automated detection and judgment process, improves the accuracy of equipment operational status identification and is not affected by subjective human factors. Moreover, because this solution adopts a non-contact detection mode, it eliminates the need for operators to perform anomaly detection on the device under test at close range, thus significantly reducing on-site operational risks and fully ensuring the personal safety of operators. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of the structure of an imager provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a thermal imaging module provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a main body and an acoustic signal acquisition module provided in an embodiment of this application; Figure 4 This is a schematic diagram of another thermal imaging module provided in an embodiment of this application; Figure 5 This is a schematic diagram of another main body and acoustic signal acquisition module provided in an embodiment of this application; Figure 6 This is an exploded schematic diagram of an imager provided in an embodiment of this application; Figure 7 This is a schematic diagram of another imager provided in an embodiment of this application; Figure 8 This is a schematic diagram of another imager provided in an embodiment of this application; Figure 9 This is a schematic diagram of another imager provided in an embodiment of this application; Figure 10 This is an exploded view of an acoustic signal acquisition module provided in an embodiment of this application; Figure 11 This is a schematic diagram of another imager provided in an embodiment of this application; Figure 12 This is a flowchart of a device operating status identification method provided in an embodiment of this application; Figure 13 This is a schematic diagram of a type of partial discharge provided in an embodiment of this application; Figure 14 This is a schematic diagram of an imager performing multimodal visualization according to an embodiment of this application; Figure 15 This is a schematic diagram of another imager used for multimodal visualization according to an embodiment of this application; Figure 16 This is a schematic diagram of the structure of a device for identifying the operating status of an application provided in an embodiment of this application.

[0040] Legend 1. First imaging module; 11. Acoustic-permeable diaphragm panel; 12. Microphone array board; 13. FPGA board; 2. Second imaging module; 21. Second snap-fit ​​structure; 22. Second positioning structure; 211. Limiting groove structure; 3. Main body; 301. Housing; 302. Functional unit; 303. Display screen; 301a, First shell portion; 301b, Second shell portion; 3011, Connecting part; 3012, Grip part; 3013, Battery cover; 3014, Drive button; 3015, Pin; 30130, First latch; 30131, Receiving groove; 30140, Second latch; 100, Battery mounting groove; 31. First snap-fit ​​structure; 32. First positioning structure; 311. Limiting protrusion structure; 4. Battery. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0042] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items that have essentially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor does it limit the quantity or execution order. It should also be understood that although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms.

[0043] These terms are simply used to distinguish one element from another. For example, without departing from the various examples, the first element can be referred to as the second element, and similarly, the second element can be referred to as the first element. Both the first and second elements can be elements, and in some cases, they can be separate and distinct elements.

[0044] "At least one" refers to one or more elements. For example, at least one element can be one element, two elements, three elements, or any integer number of elements greater than or equal to one. "Multiple" refers to two or more elements. For example, multiple elements can be two elements, three elements, or any integer number of elements greater than or equal to two.

[0045] In this article, "and / or" indicates that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0046] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant regions.

[0047] This application provides a solution for identifying the operational status of equipment (also known as anomaly detection) based on an imager. The application areas of this solution include, but are not limited to, power line applications, petrochemical applications, aircraft maintenance applications, and building inspection applications; this application does not limit these areas. In other words, operators can use the imager to perform partial discharge detection on power equipment, gas and liquid leak detection on pressure-bearing equipment, and fault detection on mechanical structures (such as mechanical friction and structural overheating).

[0048] Example 1: Taking partial discharge detection as an example, operators can use an imager to identify the operating status or detect anomalies of power equipment to identify whether the power equipment is malfunctioning. Power equipment includes, but is not limited to: transformers, switchgear, GIS (Gas Insulated Substation) switchgear, cable joints, insulators, reactors, etc., which are not limited in this application.

[0049] Example 2: Taking a gas and liquid leak detection scenario as an example, operators can use an imaging device to detect whether gas or liquid leaks occur in pressure-bearing equipment, thus meeting the safety management requirements of pressure-bearing equipment. Pressure-bearing equipment includes, but is not limited to: pressure vessels, pressure pipelines, gas storage tanks, chemical reaction vessels, gas pipelines, hydraulic pipelines, etc., which are not limited in this application.

[0050] Example 3: Taking structural acoustic anomaly detection as an example, operators can use an imaging device to detect faults in mechanical structures. The mechanical structures mentioned here include, but are not limited to, various rotating or transmission mechanical structures such as fans, pumps, reducers, motors, conveyor belts, gearboxes, and rolling mills; this application does not limit this.

[0051] The first point to note is that, in addition to the scenarios listed above, this solution can be extended to other fields. That is, this solution is applicable to a variety of industrial equipment exhibiting acoustic, thermal, or electrical anomalies.

[0052] The second point to note is that the solution for identifying the operating status of a device based on an imager provided in this application has been improved at both the hardware and software levels.

[0053] In terms of physical structure, the imager provided in this application embodiment is a portable acoustic imager. It uses a lightweight shell and an internal detachable screw locking mechanism, resulting in a small overall weight and meeting portability requirements; that is, the imager described herein is a handheld design. Furthermore, the imager has a slot-type structure that supports quick insertion and removal of the thermal imaging module, such as a pre-reserved thermal imaging slot inside the casing. Because the thermal imaging module uses a slot-type detachable design, operators can quickly insert and remove the thermal imaging module on-site, enabling rapid replacement or upgrades of the thermal imaging module.

[0054] The imager provided in this application embodiment can simultaneously acquire optical signals (corresponding to visible light images), acoustic signals (corresponding to acoustic imaging maps), and thermal imaging signals (corresponding to thermal imaging maps). Therefore, in terms of software logic, based on the imager, the following can be achieved: automatic identification of the scene to be detected, identification and output of equipment operating status, real-time plotting of PRPD maps, multimodal visualization based on at least one of acoustic imaging maps, thermal imaging maps, visible light images, or PRPD maps, analysis of operating status identification results through a large language model, and support for interactive question-and-answer sessions between operators and the imager based on the operating status identification results. Furthermore, this application embodiment also supports automatic generation of end-to-end equipment status reports and remote transmission of reports.

[0055] Based on the above description, in terms of physical structure, the imager includes at least: a visible light imaging module, a thermal imaging module, an acoustic signal acquisition module, and a main body (such as including an FPGA board and / or a core processing board). In terms of software logic, the imager includes at least: an AI (Artificial Intelligence) algorithm model library, an analysis and interaction module, and a report generation module, to collaboratively complete the signal processing flow described below, thereby realizing the identification of the device's operating status.

[0056] The physical structure of the imager will be described in detail first, followed by a detailed explanation of the methods for controlling the operation of the imager-based device.

[0057] Figure 1 This is a schematic diagram of the structure of an imager provided in an embodiment of this application. See also... Figure 1 The imager includes: an acoustic signal acquisition module 1, a thermal imaging module 2, and a main body 3.

[0058] The acoustic signal acquisition module 1 is fixedly connected to the main body 3, the thermal imaging module 2 is snapped into the main body 3, and the main body 3 is electrically connected to the acoustic signal acquisition module 1 and the thermal imaging module 2 respectively.

[0059] In some embodiments, the main body 3 and the thermal imaging module 2 are connected by a snap-fit ​​connection that facilitates easy assembly and disassembly. The thermal imaging module 2 can be used as an extended imaging module. When both acoustic imaging and thermal imaging are required, the thermal imaging module 2 can be easily and quickly snapped onto the main body 3. When acoustic imaging and thermal imaging are not required, the thermal imaging module 2 can be easily and quickly detached from the main body 3, allowing the imager to be used as a standalone imager for acoustic signal acquisition, thus expanding the application scenarios applicable to the imager.

[0060] The specific structure of the imager will be described below.

[0061] In some embodiments, the main body 3 and the thermal imaging module 2 are engaged through the cooperation of protrusions and grooves. For example... Figure 2 and Figure 3 As shown, the outer wall of the main body 3 has a first snap-fit ​​structure 31, and the outer wall of the thermal imaging module 2 has a second snap-fit ​​structure 21. The first snap-fit ​​structure 31 has a groove structure, and the second snap-fit ​​structure 21 has a protrusion structure. The first snap-fit ​​structure 31 and the second snap-fit ​​structure 21 are compatible.

[0062] The first latching structure 31 can be located on the top of the main body 3, while the second latching structure 21 can be located on the top of the thermal imaging module 2. That is, when the operator grips the imager to locate the fault point, the thermal imaging module 2 is located above the main body 3. Alternatively, the first latching structure 31 can be located on the side of the main body 3, while the second latching structure 21 can be located on the side of the thermal imaging module 2. That is, when the operator grips the imager to locate the fault point, the thermal imaging module 2 and the main body 3 are distributed horizontally.

[0063] See Figure 3 and Figure 5 The first snap-fit ​​structure 31 can be a square groove. In some embodiments, the aspect ratio of the square groove can be in the range of 2 to 3, that is, the first snap-fit ​​structure 31 is a strip-shaped directional groove. Accordingly, see Figure 2 and Figure 4 The thermal imaging module 2 has a columnar structure with a rounded rectangular cross-section. One sidewall of this columnar structure serves as the bottom. The bottom of the thermal imaging module 2 has a second snap-fit ​​structure 21, which is a columnar structure with a square outer edge. In some embodiments, the second snap-fit ​​structure 21 and the bottom of the thermal imaging module 2 can have a smooth transition. This avoids stress concentration at the connection between the second snap-fit ​​structure 21 and the bottom of the thermal imaging module 2, thereby improving the overall strength of the thermal imaging module 2.

[0064] In other embodiments, see Figure 3 and Figure 4 The first snap-fit ​​structure 31 may also have a limiting protrusion structure 311 inside, and the second snap-fit ​​structure 21 may also have a limiting groove structure 211 on the outside, with the limiting groove structure 211 matching the limiting protrusion structure 311. In this way, through the cooperation between the limiting groove structure 211 and the limiting protrusion structure 311, the main body 3 and the thermal imaging module 2 can be securely connected, improving the connection stability between the two and making it less likely for the thermal imaging module 2 to slip off from the main body 3 in the assembled state.

[0065] In some embodiments, such as Figure 3 As shown, when the first engaging structure 31 is a groove structure, the limiting protrusion structure 311 is located at the bottom of the groove of the first engaging structure 31. The limiting protrusion structure 311 can have an annular structure, and the outer wall of the limiting protrusion structure 311 is adapted to the shape of the groove wall of the first engaging structure 31, that is, the limiting protrusion structure 311 is a square annular protrusion structure, and the outer wall of the limiting protrusion structure 311 is spaced apart from the groove wall of the first engaging structure 31. Correspondingly, as Figure 4 As shown, the limiting groove structure 211 is located at the suspended end of the second snap-fit ​​structure 21, and the limiting groove structure 211 is adapted to the limiting protrusion structure 311.

[0066] In some embodiments, the limiting protrusion structure 311 is located on the groove wall of the first snap-fit ​​structure 31, and the limiting protrusion structure 311 may have a columnar structure. Correspondingly, the limiting groove structure 211 is located on the side wall of the second snap-fit ​​structure 21, and the limiting groove structure 211 is adapted to the limiting protrusion structure 311.

[0067] In some embodiments, the limiting protrusion structure 311 and the limiting groove structure 211 may be a transition fit or an interference fit.

[0068] In this way, the cooperation between the limiting protrusion structure 311 and the limiting groove structure 211 can prevent the second snap-fit ​​structure 21 from rotating within the first snap-fit ​​structure 31, thereby limiting the relative rotation between the main body 3 and the thermal imaging module 2.

[0069] In other embodiments, the top of the main body 3 has a first snap-fit ​​structure, and the bottom of the thermal imaging module 2 has a second snap-fit ​​structure. The first snap-fit ​​structure is a protruding structure, and the second snap-fit ​​structure is a groove structure, and the first snap-fit ​​structure and the second snap-fit ​​structure are adapted to each other. Further, the outer side of the first snap-fit ​​structure may also have a limiting groove structure, and the inner side of the second snap-fit ​​structure may also have a limiting protrusion structure, with the limiting groove structure adapted to the limiting protrusion structure. For the specific shapes and arrangements of the first snap-fit ​​structure, the second snap-fit ​​structure, the limiting protrusion structure, and the limiting groove structure, please refer to the above description, which will not be repeated here.

[0070] In some embodiments, see Figure 2 and Figure 3 The top of the main body 3 has a first positioning structure 32, and the bottom of the thermal imaging module 2 has a second positioning structure 22. The first positioning structure 32 is a groove structure, and the second positioning structure 22 is a protrusion structure, and the second positioning structure 22 is adapted to the first positioning structure 32.

[0071] See Figure 2 and Figure 3 The second positioning structure 22 can be located on the side of the second snap-fit ​​structure 21 near the main body 3. The second positioning structure 22 can be a cylindrical structure, with the end of the second positioning structure 22 near the main body 3 being hemispherical and smoothly transitioning to the cylindrical structure. The first snap-fit ​​structure 31 is a groove structure, and the first positioning structure 32 is located at the bottom of the groove of the first snap-fit ​​structure 31. The first positioning structure 32 can be a circular groove, and the circular groove and the cylindrical structure can be fitted with a clearance fit or a transition fit.

[0072] During implementation, when the operator installs the thermal imaging module 2, they can first align the first positioning structure 32 with the second positioning structure 22 so that the thermal imaging module 2 and the mounting structure of the main body 3 are positioned. Then, the operator can press down on the thermal imaging module 2 to complete the assembly, thus improving the ease of assembly.

[0073] In some embodiments, see Figure 2 and Figure 3 The thermal imaging module 2 has a connector plug on the side near the main body 3, and the connector plug extends in the same direction as the second positioning structure 22. The top of the main body 3 has a first snap-fit ​​structure 31, which is a groove structure. A connector socket is located at the bottom of the groove of the first snap-fit ​​structure 31. The thermal imaging module 2 and the main body 3 are electrically connected through the cooperation between the connector plug and the connector socket. In some embodiments, the connector plug is a TYPE-C connector plug, and correspondingly, the connector socket is a TYPE-C connector socket.

[0074] In some embodiments, such as Figure 6 As shown, the main body 3 includes a housing 301, a core processing board 302, and a display screen 303. The core processing board 302 is located inside the housing 301 and is electrically connected to the acoustic signal acquisition module 1, the thermal imaging module 2, and the display screen 303, respectively.

[0075] The display screen 303 is located on the side of the housing 301 away from the acoustic signal acquisition module 1.

[0076] See Figure 6 And refer to Figure 3 The housing 301 includes a first housing portion 301a and a second housing portion 301b. The first housing portion 301a is the front housing portion, and the second housing portion 301b is the rear housing portion. The first housing portion 301a and the second housing portion 301b can be connected by screws, and the two enclose a receiving space. The core processing board 302 is located in the receiving space, and the core processing board 302 can be connected to the first housing portion 301a and / or the second housing portion 301b by screws.

[0077] In implementation, the first housing 301a is used to connect the acoustic signal acquisition module 1, and the first housing 301a may be provided with a tripod nut and a shoulder strap anchor point (neither shown in the figure). The tripod nut provides assembly space for the tripod. After the imager is assembled with the tripod, the operator can use the tripod to achieve stable support for the imager, improving the ease of operation of the imager. The shoulder strap anchor point provides installation space for the shoulder strap. After the imager is assembled with the shoulder strap, the operator can use the shoulder strap to hang the imager on their back, improving the ease of carrying the imager. The second housing 301b is provided with operation buttons on the side away from the acoustic signal acquisition module 1. In some embodiments, the above-mentioned operation buttons include, but are not limited to: photo or video recording buttons, power buttons, fill light buttons, direction control buttons, confirmation buttons, return buttons, menu buttons, etc., to facilitate the operator to quickly operate the device in different scenarios. The second housing 301b typically also includes a power / transmission interface, headphone jack, TF card interface, SIM card interface, external speaker interface, analog microphone interface, thermal imaging module slot, and so on.

[0078] In some embodiments, see Figure 7 The housing 301 includes a connecting part 3011 and a gripping part 3012.

[0079] The top of the connecting part 3011 is engaged with the thermal imaging module 2, and the holding part 3012 is located on the side of the connecting part 3011 away from the thermal imaging module 2 and is connected to the connecting part 3011.

[0080] In this way, the thermal imaging module 2 is located on the upper side of the connecting part 3011. Compared with the technical solution where the thermal imaging module 2 is installed on the side of the main body 3, the thermal imaging module 2 is less likely to slip off during the use of the imager. The grip part 3012 is located on the lower side of the connecting part 3011, which allows the center of gravity of both the connecting part 3011 and the thermal imaging module 2 to be located above the grip part 3012, reducing the difficulty for the operator to hold the imager.

[0081] In other embodiments, see Figure 7 The grip portion 3012 has a columnar structure, and in the axial direction of the grip portion 3012, the projection of the grip portion 3012 at least partially overlaps with the projection of the thermal imaging module 2.

[0082] This allows the centers of gravity of the thermal imaging module 2 and the grip 3012 to be aligned in the same vertical direction as much as possible, further reducing the difficulty for operators to hold the imager.

[0083] In some embodiments, such as Figure 8As shown, the first normal of the working surface of the acoustic signal acquisition module 1 is coplanar with the second normal of the working surface of the thermal imaging module 2. In this way, the optical axis of the thermal imaging module is coplanar with the central axis of the acoustic array. When displaying the acoustic and thermal images using an imager, secondary alignment is not required on-site, thus improving detection accuracy.

[0084] In some embodiments, the acoustic signal acquisition module 1 and the main body 3 can be connected by one of a variety of connection methods, such as screw connection, snap connection, or riveting.

[0085] See Figure 10 The acoustic signal acquisition module 1 includes a sound-transparent diaphragm panel 11, a microphone array board 12, and an FPGA board 13 (Field Programmable Gate Array).

[0086] See Figure 10 And refer to Figure 7 The acoustic membrane panel 11 has multiple spaced acoustic holes, and the acoustic membrane panel 11 is connected to the first shell portion 301a of the housing 301 by multiple screws. This ensures that the connection between the acoustic membrane panel 11 and the housing 301 is sufficiently secure, and also allows for quick disassembly when maintenance or replacement is required, improving the ease of assembly and disassembly.

[0087] In some embodiments, the acoustic membrane panel 11 is connected to the housing 301 by at least eight M3 screws. The acoustic membrane panel 11 is made of a material with good acoustic transparency, physical isolation, and environmental resistance. The specific material of the acoustic membrane panel 11 is not limited in the embodiments of this application.

[0088] The microphone array board 12 has approximately two hundred digital microphones, which are evenly distributed on one side of the microphone array board 12 near the acoustic diaphragm panel 11 to capture acoustic signals. The microphone array board 12 is connected to the first housing portion 301a of the housing 301 by multiple screws. This ensures a sufficiently secure connection between the microphone array board 12 and the housing 301, and also allows for quick disassembly when maintenance or replacement is required, improving ease of assembly and disassembly.

[0089] In some embodiments, the microphone array board 12 is connected to the housing 301 by at least four M3 screws.

[0090] See Figure 10 And refer to Figure 6The FPGA board 13 is located on the side of the microphone array board 12 away from the acoustic diaphragm panel 11. The FPGA board 13 is electrically connected to both the microphone array board 12 and the core processing board 302. In practice, the FPGA board 13 can acquire approximately two hundred acoustic signals from the microphone array board 12 within milliseconds, process the acquired acoustic signals accordingly, and then send them to the core processing board 302 of the main body 3.

[0091] In some embodiments, the FPGA board 13 is connected to the first housing portion 301a of the housing 301 by a plurality of screws, and the FPGA board 13 and the housing 301 are connected by at least four M3 screws.

[0092] In practice, the acoustic signal passes through the acoustic diaphragm panel 11 and is partially filtered by the acoustic diaphragm panel 11 before being captured by the microphone array board 12. The ADC (Analog-to-Digital Converter) integrated in the microphone array board 12 converts the acoustic signal into a digital signal and sends the digital signal to the FPGA board 13. The FPGA board 13 processes these digital signals at high speed and sends the processed digital signal to the core processing board 302.

[0093] In this embodiment, since the FPGA board 13 and the core processing board 302 are designed independently, the core processing board 302 undertakes all signal processing work except for the FPGA board 13. After receiving the digital signals transmitted by the FPGA board 13, the core processing board 302 completes subsequent signal analysis and feature calculation by running various algorithms, ultimately realizing the identification of the operating status of the device under test. In addition, considering the battery life of the device, both the core processing board 302 and the FPGA board 13 are selected as low-power products.

[0094] In this embodiment, the core processing board 302 also has a storage function, such as storing the generated acoustic and thermal images. Additionally, the core processing board 302 can be electrically connected to the imager's external storage unit (such as an SD card) and communication module. The core processing board 302 stores the acquired data, intermediate data obtained after data processing, or result data in the external storage unit, facilitating operators to disassemble the external storage unit and further analyze the stored data using computer equipment. Alternatively, the core processing board 302 can transmit the acoustic and thermal images externally via the communication module, improving transmission convenience.

[0095] In some embodiments, such as Figure 9As shown, the grip portion 3012 has a columnar structure. The first end of the grip portion 3012 is connected to the connecting portion 3011. The second end of the grip portion 3012 has a battery mounting groove 100. The battery mounting groove 100 can be a square groove, and the battery mounting groove 100 extends along the axial direction of the grip portion 3012.

[0096] The imager also includes a battery 4, which can be a rechargeable cyclic battery. The battery 4 has a cubic structure and is interference-fitted with the battery mounting slot 100.

[0097] Furthermore, housing 301 also includes battery cover 3013. See also Figure 9 The battery cover 3013 is located on the side of the grip portion 3012 away from the connecting portion 3011. The side of the battery cover 3013 near the grip portion 3012 has a receiving groove 30131. When the battery cover 3013 is connected to the grip portion 3012, the battery cover 3013 is used to cover the opening of the battery mounting groove 100.

[0098] In this way, when the operator uses the imager to locate the fault point, after the battery 4 is depleted, the battery cover 3013 can be opened and the battery 4 can be replaced. Since the battery 4 and the battery mounting slot 100 are interference fit, even if the opening of the battery mounting slot 100 is facing down after the battery cover 3013 is opened, the battery 4 will not fall off by itself under the action of friction, thus avoiding the possibility of the battery 4 falling off and being damaged.

[0099] In some embodiments, see Figure 9 The length of the battery 4 is greater than the depth of the battery mounting slot 100, so that after the battery 4 is assembled into the battery mounting slot 100, at least a portion of the battery 4 is located outside the opening of the battery mounting slot 100. Accordingly, the battery cover 3013 has a receiving groove 30131 on the side near the grip portion 3012. The battery cover 3013 is used not only to cover the opening of the battery mounting slot 100, but also to receive the portion of the battery 4 that has shifted outside the battery mounting slot 100.

[0100] Since the length of battery 4 is greater than the depth of battery mounting slot 100, at least part of battery 4 is exposed outside the slot opening of battery mounting slot 100 after battery cover 3013 is removed, which makes it convenient for operators to grasp and remove battery 4 and improves the ease of replacing battery 4.

[0101] In some embodiments, the length of the battery 4 is less than the depth of the battery mounting groove 100. That is, after the battery 4 is assembled into the battery mounting groove 100, the battery 4 is completely located inside the battery mounting groove 100, and there is a receiving space between the end face of the battery 4 near the opening of the battery mounting groove 100 and the opening.

[0102] Furthermore, the imager also includes a pull ring structure (not shown), which is hinged to the end face of the battery 4 near the slot opening. The thickness of the pull ring structure is no greater than the difference between the depth of the battery mounting slot 100 and the length of the battery 4, and the length of the pull ring structure is greater than the difference between the depth of the battery mounting slot 100 and the length of the battery 4. Thus, when the pull ring structure rotates to be parallel to the end face of the battery 4 near the slot opening, it can be completely housed within the aforementioned receiving space. It is easy to understand that as the pull ring structure gradually rotates, the angle between it and the end face of the battery 4 near the slot opening gradually increases. When the pull ring structure rotates to be perpendicular to the end face of the battery 4 near the slot opening, the pull ring structure is at least partially outside the slot opening. At this point, the operator can manually pull the pull ring structure and remove the battery 4, improving the convenience of replacing the battery 4.

[0103] In some embodiments, the length of the battery 4 is equal to the depth of the battery mounting slot 100. That is, after the battery 4 is assembled into the battery mounting slot 100, the battery 4 is completely located inside the battery mounting slot 100, and the end face of the battery 4 near the opening of the battery mounting slot 100 is flush with the opening.

[0104] Furthermore, the imager may also include a drive mechanism, which may include a button and a transmission component (both not shown). The button is disposed on the grip portion 3012, and the transmission component is disposed inside the grip portion 3012. The transmission component is connected to the button and the wall surface of the battery 4 away from the opening of the battery mounting slot 100, respectively. The transmission component is configured to push at least a portion of the battery 4 away from the battery mounting slot 100 when the button is pressed. In this way, when the battery 4 is depleted, the battery cover 3013 can be opened, and then the button can be pressed to extend at least a portion of the battery 4 out of the battery mounting slot 100, and finally the battery 4 can be removed and replaced, thus improving the convenience of battery replacement.

[0105] In some embodiments, one end of the battery cover 3013 is hinged to the grip portion 3012, such as... Figure 9 As shown, the housing 301 also includes a pin 3015. A first hinge seat is provided at one end of the battery cover 3013, and a second hinge seat is provided at a corresponding position on the grip portion 3012. The first and second hinge seats are hinged together by the pin 3015. (Continue to see...) Figure 9The other end of the battery cover 3013 is engaged with the grip portion 3012. The battery cover 3013 has a first latch 30130 on the side near the grip portion 3012. The housing 301 also includes a drive button 3014, which is disposed on the side wall of the grip portion 3012 and is throttle-connected to it. A second latch 30140 is disposed on the side of the grip portion 3012 near the battery cover 3013 and is connected to the drive button 3014. The first latch 30130 and the second latch 30140 cooperate to achieve the engagement between the battery cover 3013 and the grip portion 3012. Optionally, the housing 301 includes an elastic reset member, which can be a spring. The elastic reset member is connected to the second latch 30140 and provides a pressing force to the second latch 30140 so that the drive button 3014 resets after being released from pressure.

[0106] In practice, when the operator needs to replace the battery 4, he can press the drive button 3014 to move the second latch 30140 to release the engagement between the second latch 30140 and the first latch 30130, thereby allowing the other end of the battery cover 3013 to separate from the grip part 3012, exposing the battery 4 for subsequent replacement operations.

[0107] In some embodiments, the communication module includes both wireless and wired communication interfaces, which is not limited in this application. The wireless communication interface encompasses types such as WiFi (Wireless Fidelity), xth generation mobile communication technology, or Bluetooth. The wired communication interface includes, but is not limited to, an Ethernet port or video streaming interface, and supports USB flash drive storage and charging functions. The wireless communication module is responsible for wireless data transmission between the imager and external devices or servers, meeting the needs of remote monitoring and data sharing. The wired communication interface offers advantages such as high transmission rate and strong stability, enabling high-speed data transmission and also accommodating device charging and external storage unit connection.

[0108] In some embodiments, the imager further includes a visible light imaging module. For example... Figure 9 As shown, the visible light imaging module includes a camera and two supplementary lights. The visible light module is located on the side of the housing 301 furthest from the display screen 303. For example, the visible light module can be mounted on the microphone array board 12; this application is not limited to this. In this embodiment, the visible light module acquires visible light images by converting the light signal into an electrical signal and transmitting it to the core processing board 302 for subsequent processing. Furthermore, in low-light environments, the supplementary lights can be automatically or manually activated to enhance illumination, thereby ensuring high-quality visible light images can be acquired in various environments.

[0109] In other embodiments, the imager also includes a power module. The power module employs a removable battery power supply, with flexible charging options supporting both Type-C interface charging and battery dock charging. Additionally, considering the device's battery life, a lithium battery can be used; this application does not limit this choice. The removable design allows for quick battery replacement, enabling replacement before the battery is depleted, fundamentally preventing equipment downtime due to insufficient power and ensuring continuous operation of the equipment's testing tasks.

[0110] This application provides an imager with an embodiment in which the main body and thermal imaging module are connected by a snap-fit ​​connection that facilitates easy assembly and disassembly. The thermal imaging module can be used as an extended imaging module. In situations where both acoustic and thermal imaging are required, the thermal imaging module can be easily and quickly snapped onto the main body. When thermal imaging is not required, the thermal imaging module can be easily and quickly detached from the main body, allowing the imager to be used as a standalone acoustic imaging device, thus expanding its applicable scenarios. In other words, because the thermal imaging module uses a slot-type detachable design, operators can quickly insert and remove the thermal imaging module on-site, enabling rapid replacement or upgrades. This design supports both pure acoustic mode and acoustic-thermal mode. Furthermore, the imager uses a lightweight shell and an internal detachable screw locking mechanism, resulting in a small overall weight and meeting portability requirements.

[0111] In summary, this application provides a portable, modular, plug-and-play thermal imaging module that can be quickly deployed, has low power consumption, and long battery life. This device features a lightweight body, supports handheld operation, and has a plug-and-play thermal imaging module. Based on the imager provided in this application, a workflow can be implemented for data acquisition and processing, result display, automatic report generation, and remote transmission and storage. The workflow of the imager is described in detail below.

[0112] Based on the aforementioned imager, this application embodiment also provides a device operation status identification method. The execution subject of this method is the aforementioned imager, such as the device operation status identification component of the imager. In this application embodiment, the device operation status identification component includes the FPGA board 13 and the core processing board 302 of the aforementioned imager. Figure 12 This is a flowchart of a device operating status identification method provided in an embodiment of this application. See also... Figure 12 The method includes the following steps.

[0113] 1201. The imager acquires acoustic signals collected in the target scene.

[0114] It should be noted that if the imager is currently equipped with a thermal imaging module, the imager will also acquire the thermal imaging signal radiated by the device under test in the target scene.

[0115] In this embodiment, the target scene is also referred to as the scene to be tested. After the operator holds the imager to the scene to be tested, the imager can automatically identify the scene and call the corresponding algorithm model in the AI ​​algorithm model library to identify the operating status of the device under test and display it visually. That is, after the operator holds the imager to the scene to be tested, the visible light module of the imager will collect a visible light image of the scene to be tested, i.e., take a picture of the scene to be tested, and obtain a real-world image of the scene to be tested. The FPGA board built into the imager will extract and recognize features from the visible light image to identify the appearance of the device, understand the scene, and mark the position of the device under test in the scene to be tested. Taking the device under test in the scene to be tested as a transformer as an example, the imager will call the power partial discharge recognition model in the AI ​​algorithm model library to identify the operating status of the device under test and display it visually.

[0116] In addition, the imager will prompt the operator to specify the exact detection location of the device under test, so that the imager can collect acoustic signals at the specific detection location of the device under test. The imager may prompt the operator in the following ways: displaying corresponding text on the screen or broadcasting voice prompts, which is not limited in this application.

[0117] In this embodiment, the acoustic signal acquisition module of the imager is responsible for acquiring acoustic signals from various directions in the scene to be tested. These acoustic signals include ultrasonic signals (such as 100kHz–500kHz partial discharge sound) and / or audible sound signals (such as gas leak noise, mechanical noise, etc.), thereby forming multi-channel raw acoustic data. Additionally, if the imager is equipped with a thermal imaging module, the thermal imaging module captures the infrared radiation signal emitted from the surface of the device under test. After the infrared radiation is focused by the infrared lens and converted into photoelectric signals by the focal plane array, a raw infrared electrical signal, i.e., the thermal imaging signal, is generated.

[0118] It should be noted that the embodiments of this application also involve clock synchronization (time alignment) and spatial mapping (position alignment). That is, the FPGA board or core processing board synchronizes the clocks of the visible light module, thermal imaging module, and acoustic signal acquisition module, and establishes a spatial mapping relationship between the visible light image coordinates (pixel coordinates in the visible light image, from image space) – thermal imaging coordinates (pixel coordinates in the thermal image, from thermal imaging space) – and acoustic spatial coordinates (sound source position coordinates, from detection space) based on factory calibration parameters. This ensures that subsequent acoustic-thermal-visual data can be fused and displayed in the same coordinate system, achieving accurate overlay presentation of fault location and features. The purpose of clock synchronization is to unify the acquisition time of the visible light, thermal imaging, and acoustic modules, ensuring that the three modules acquire data at exactly the same time point. That is, the FPGA board or core processing board sends a unified clock signal to the three modules, ensuring that they acquire data simultaneously at t=0. The establishment of the spatial mapping relationship is the process of unifying the visible light image coordinates, thermal imaging coordinates, and acoustic spatial coordinates to the same reference coordinate system based on factory calibration parameters.

[0119] The following describes how the imager outputs prompts to the operator, guiding them to align the acoustic signal acquisition with the specific detection location of the device under test.

[0120] 1201-1. Obtain a real-world image of the target scene and perform target detection on the real-world image to obtain the target detection result; the target detection result includes the device type and location coordinates of the device under test.

[0121] Here, the real-world image of the target scene refers to the visible light image acquired by the imager in the scene to be detected. Before performing target detection on the visible light image, preprocessing is performed on the visible light image to improve the accuracy of subsequent target detection. In some embodiments, the preprocessing of the visible light image includes, but is not limited to, distortion correction, noise suppression, brightness or contrast enhancement, etc., which are not limited in this application.

[0122] In some embodiments, the imager performs target detection on the visible light image based on a target detection model (such as a convolutional neural network) to detect and identify device targets included in the visible light image, and then outputs the bounding box (including location coordinates) and category label (to indicate the device type) of the detected device. Through this step, the imager can distinguish whether the device under test in the scene is a transformer, switch cabinet, pressure vessel, pipeline, valve, or mechanical structure, etc.

[0123] 1201-2. Perform thermal anomaly detection based on thermal imaging images to obtain thermal anomaly detection results; wherein, the thermal anomaly detection results include the contour coordinates of the detected thermal anomaly region.

[0124] This step includes thermal imaging signal processing, temperature field reconstruction, and thermal anomaly detection.

[0125] In the thermal imaging signal processing and temperature field reconstruction stage, the weak infrared electrical signal output by the focal plane array (FPA) is first amplified and filtered to effectively suppress interference noise caused by background radiation and improve signal quality. Then, by combining the detector response curve and the emissivity parameters of the device under test, the infrared electrical signal is accurately converted into pixel temperature values ​​to achieve temperature calibration. Finally, a pseudo-color mapping method is used to convert different temperature ranges into differentiated color representations to generate a two-dimensional thermal image, complete the temperature field reconstruction, and intuitively show the temperature distribution characteristics of the surface of the device under test.

[0126] In the thermal anomaly detection stage, the thermal image is input into the thermal anomaly detection algorithm. By comprehensively utilizing threshold segmentation, connected component analysis, and / or a deep learning-based hot spot detection model, the algorithm can accurately identify thermal anomaly areas such as local overheating and hot spot aggregation, and output the contour coordinates and key temperature features of the thermal anomaly area.

[0127] 1201-3. Based on the target detection results and thermal anomaly detection results, determine the candidate detection area of ​​the device under test; mark the candidate detection area on the real scene map and thermal imaging map, and output prompt information; wherein, the prompt information is used to prompt the imager to be aligned with the candidate detection area.

[0128] In this embodiment of the application, the imager can automatically generate one or more candidate detection areas based on the target detection results and thermal anomaly detection results, combined with the preset prior knowledge of the equipment structure (such as transformer bushings, pressure vessel welds, mechanical bearing positions, etc.), and visually mark them on the visible light image and thermal image.

[0129] In addition, the imager can convert the image pixel coordinates of the candidate detection area into acoustic spatial coordinates and beam pointing parameters in the detection space of the acoustic signal acquisition module, according to pre-calibrated coordinate transformation rules. That is, after automatically selecting suspected abnormal areas in the image, the coordinate transformation rules complete the mapping from image space to acoustic space, converting the suspected abnormal areas into acoustic spatial coordinates or beam pointing parameters recognizable by the acoustic signal acquisition module. This achieves precise matching between the suspected abnormal areas and the acoustic detection area, providing data support for subsequent precise acoustic directional detection. For example, the imager can send beam pointing angle parameters to the microphone array, guiding the microphone array to precisely align the ultrasonic detection beam with the suspected abnormal area, directionally acquiring the ultrasonic signal in that area. Simultaneously, when visually displaying visible light images and thermal images on the screen, candidate detection areas can be highlighted and prompts can be output simultaneously. For example, through voice broadcasting, operators can be prompted to align the detection direction with suspected abnormal locations such as transformer high-voltage bushings.

[0130] 1202. The imager uses the device type of the device under test as a constraint condition to generate at least one image based on acoustic signals.

[0131] It should be noted that if the imager is currently equipped with a thermal imaging module, the imager will generate an image based on one or more of the acoustic signals or thermal imaging signals, using the device type of the device under test as a constraint. Additionally, if the device under test is an electrical device, at least one of the generated images will include a PRPD map, which is used to describe the partial discharge characteristics of the device under test through phase distribution.

[0132] In this embodiment, the imager invokes a machine learning model from the AI ​​algorithm model library that matches the device type of the device under test to perform the step of generating at least one image based on acoustic signals and thermal imaging signals. For example, when the device type is a transformer, the partial discharge identification model in the AI ​​algorithm model library is invoked; when the device type is a pressure vessel or pressure pipeline, the gas and liquid leakage identification model in the AI ​​algorithm model library is invoked; and when the device type is a mechanical structure, the structural acoustic anomaly identification model in the AI ​​algorithm model library is invoked.

[0133] It should be noted that the AI ​​algorithm model library includes algorithms that, in addition to generating at least one image, are also responsible for identifying the operating status of the equipment under test, outputting the operating status identification results, and visualizing them based on at least one image. For example, when the equipment under test is a pressure vessel or pressure pipeline, the gas-liquid leakage identification model is called to identify gas-liquid leaks, output the identification results, and present them in a multimodal visualization. Similarly, when the equipment under test is a mechanical structure, the structural acoustic anomaly identification model is used to identify structural acoustic anomalies, output the identification results, and present them in a multimodal visualization.

[0134] In some embodiments, at least one image is generated based on one or more of acoustic signals or thermal imaging signals, including steps 1202-1 to 1202-4.

[0135] 1202-1. Real-time generation of PRPD maps based on acoustic signals.

[0136] It should be noted that, in response to the target detection result in step 1201-1 above including the equipment type of the tested device and indicating that the tested device is an electrical device, the imager will generate a PRPD map in real time based on the acoustic signal. When the tested device is a pressure vessel or pressure pipeline or other pressure-bearing equipment or mechanical structure, the imager will not generate a PRPD map.

[0137] The power supply voltage of a power system is a sinusoidal alternating current of 50Hz or 60Hz, which varies periodically, with each cycle corresponding to a 360° voltage phase. When partial discharge occurs in a device, pulses are not generated randomly, but rather concentrated within a specific phase interval of the voltage cycle, manifesting as single pulses or dense pulse clusters; and the higher the intensity of the discharge pulse, the greater the amplitude of the charge transfer it generates. The partial discharge phase phase diagram (PRPD) can visually present the correspondence between the charge amplitude of the discharge pulse and the voltage phase. In this embodiment, a PRPD diagram can be generated by extracting the ultrasonic frequency band signal from the acoustic signal, thereby enabling the determination of the partial discharge type of the device under test.

[0138] Among them, such as Figure 13 As shown, the types of partial discharge include, but are not limited to: no discharge phenomenon, floating discharge, surface discharge or corona discharge, and this application does not limit them.

[0139] The generation process of PRPD maps is described below.

[0140] First, the acoustic signal is preprocessed and its features are extracted to obtain an effective pulse signal.

[0141] Because the ultrasonic signal of partial discharge (also known as the partial discharge signal, with a frequency range of 120kHz-500kHz) is easily affected by environmental noise, equipment mechanical vibration noise, etc., it is necessary to improve the signal purity through preprocessing steps. In the embodiments of this application, the preprocessing steps include, but are not limited to, filtering, signal enhancement, and abnormal signal removal, and this application does not limit these steps. The filtering, signal enhancement, and abnormal signal removal are described below.

[0142] 1. Filtering process.

[0143] When performing filtering, one or more of the following methods can be selected.

[0144] Bandpass filtering: Preserves the target frequency band (such as the 100kHz-500kHz band of partial discharge signals) to suppress mechanical vibration noise below 50kHz and electromagnetic interference above 1MHz.

[0145] Adaptive filtering: For dynamically changing interference (such as sudden noise at the scene), the interference components are estimated in real time through the reference channel, and a reverse cancellation signal is generated in real time to cancel the influence of the interference on the partial discharge signal.

[0146] 2. When enhancing signals, peak detection and amplification, averaging and superposition, and other methods can be used to enhance periodic pulse components.

[0147] Peak detection and amplification: Given that partial discharge signals exhibit the characteristics of instantaneous pulses, with short duration and prominent peak features, a threshold can be set to filter the pulse peak signals, followed by appropriate amplification of the effective peak signals. This method can effectively improve the signal-to-noise ratio by avoiding the synchronous amplification of background noise while ensuring the effectiveness of subsequent signal analysis.

[0148] Average superposition method: The average superposition method is used to perform synchronous superposition operation on the time-domain signals acquired multiple times. By taking advantage of the periodicity of the partial discharge signal and the transformer voltage period being synchronized, and combined with the random distribution characteristics of the background noise, the amplitude of the effective discharge signal is increased and enhanced during the superposition process, while the random noise cancels each other out and weakens, thereby achieving the purification and enhancement of the effective signal.

[0149] 3. Abnormal signal rejection.

[0150] Eliminating isolated pulses: Interference signals often exhibit isolated and irregular pulse characteristics. Therefore, by judging the repeatability of the signal (such as whether it is synchronized with the voltage cycle), isolated interference pulses that appear only once can be eliminated, while recurring suspected discharge signals can be retained.

[0151] The following describes how to extract features from preprocessed acoustic signals.

[0152] In the embodiments of this application, the imager extracts one or more of time-domain features, frequency-domain features, or time-frequency-domain features. That is, feature extraction of the preprocessed acoustic signal includes at least one of the following: Temporal feature extraction is performed on the preprocessed acoustic signal; Frequency domain features are extracted from the preprocessed acoustic signal; Time-frequency domain features are extracted from the preprocessed acoustic signal.

[0153] The time-domain features, frequency-domain features, and time-frequency-domain features are introduced below.

[0154] 1. Time-domain features (features extracted directly from signal waveforms).

[0155] Pulse parameters: extract signal peak value, rise time (time from pulse start to peak value), and pulse width (pulse duration). Partial discharge pulses exhibit a narrow pulse characteristic with rapid rise and fall, such as rise time <1μs and pulse width <10μs. In contrast, mechanical vibration noise pulses have a slower rise and wider pulse width, showing a significant difference in characteristics between the two.

[0156] Statistical characteristics: Calculate the signal peak factor (the ratio of peak value to RMS value) and kurtosis (the degree of concentration of signal peaks). The kurtosis and peak factor of partial discharge signals are much higher than those of noise signals because noise signals have smoother waveforms and less prominent peak characteristics.

[0157] It should be noted that, in addition to the time-domain features mentioned above, other time-domain features can also be extracted, but this application does not make any selections in this regard.

[0158] 2. Frequency domain features (features extracted after converting the time-domain signal to the frequency domain through Fourier transform).

[0159] Spectral peaks: The spectrum of partial discharge signals has obvious characteristic peaks in the 150kHz-300kHz frequency band, while the spectrum of interference noise is mostly broadband and has no fixed characteristic peaks.

[0160] Spectral entropy: Calculates the degree of disorder in the spectrum. The spectral energy of a partial discharge signal is concentrated in a specific frequency band, resulting in a low spectral entropy value; the spectral energy of a noise signal is dispersed, resulting in a high spectral entropy value.

[0161] Frequency band energy distribution: As a core frequency domain feature, it can further distinguish the differences in the frequency band energy ratio between partial discharge signals and noise signals.

[0162] It should be noted that, in addition to the frequency domain features mentioned above, other time domain features can also be extracted, but this application does not make any selections for them.

[0163] 3. Time-frequency domain characteristics (integrating time-domain and frequency-domain characteristics to adapt to signal analysis under complex operating conditions).

[0164] By using wavelet transform and short-time Fourier transform, the signal is converted into a two-dimensional time-frequency spectrum. Partial discharge signals exhibit concentrated energy spots at specific time points and frequency bands, while noise signals show a scattered energy distribution in their time-frequency spectrum, lacking obvious concentrated characteristics.

[0165] II. Generating PRPD maps based on effective pulse signals.

[0166] 1. Acquire the amplitude and trigger timestamp of each pulse in the valid pulse signal; acquire the phase data of the power frequency voltage signal acquired synchronously with the acoustic signal. This phase data includes the phase of the power frequency voltage signal at each moment.

[0167] In this embodiment, to generate a PRPD map, the amplitude (including core acoustic features such as peak voltage and sound pressure level) and trigger timestamp of each pulse need to be extracted from the effective pulse signal. The trigger timestamp is used to correlate the pulse amplitude with the voltage phase.

[0168] Pulse amplitude: This should be used as the vertical axis of the PRPD spectrum, retaining the original signal value, such as the original electrical signal amplitude (in mV) or the original ultrasonic signal amplitude (in dB). Alternatively, a calibrated physical quantity can be used, such as the calibrated charge amplitude (in pC, obtainable via acoustic-electrical conversion). When the pulse amplitude range is small, mV can be used as the unit. When the pulse amplitude difference is large, dB can be used as the unit; this application does not impose any limitation on this.

[0169] Trigger timestamp: with a precision of microseconds, used to achieve precise matching of the pulse with the corresponding voltage phase. Synchronous voltage phase signal: Phase data of power frequency voltage (e.g., 50Hz) acquired synchronously with acoustic signals, including phase angles of 0°-360° for the entire time period, which serves as the horizontal axis reference for the PRPD spectrum.

[0170] 2. Using the zero-crossing point of the power frequency voltage signal as the zero phase, establish a mapping relationship between time and phase; based on the trigger timestamp of each pulse and the mapping relationship, determine the phase of each pulse and obtain a two-dimensional data pair for each pulse.

[0171] In this embodiment, the pulse and phase are first associated. That is, the zero-crossing point of the power frequency voltage signal (e.g., the start of the positive half-cycle) is taken as 0° phase, and a time-phase mapping relationship is established. Then, based on the trigger timestamp of each pulse and the above mapping relationship, the phase of each pulse is determined. For example, at a 50Hz power frequency, one cycle = 20ms, corresponding to 360°, so 1ms ≈ 18°. Thus, for each pulse, the corresponding phase angle can be calculated according to its trigger timestamp. Assuming a pulse is triggered 3ms after the 0° phase, its corresponding phase is 3 × 18° = 54°. Based on the above mapping relationship, a two-dimensional data pair for each pulse is obtained. For any pulse, the two-dimensional data pair includes the pulse's amplitude and phase.

[0172] 3. Generate PRPD maps based on the two-dimensional data pairs of each pulse.

[0173] For this step, the first step is to perform data statistics and construct a three-dimensional matrix of phase, amplitude, and count. To perform data statistics, it is also necessary to first divide the phase interval and the amplitude interval.

[0174] Phase interval division refers to dividing the phase into multiple phase intervals according to a preset step size. That is, the 0°-360° phase is divided into several phase intervals using a fixed step size (e.g., 1°, 2°, or 5°). A smaller step size results in higher resolution of the PRPD map. To balance accuracy and computational cost, a step size of 1°-2° is typically chosen.

[0175] Amplitude range division refers to dividing the pulse amplitude into multiple amplitude ranges according to linear or logarithmic intervals. That is, the pulse amplitude is divided into several amplitude ranges (such as linear segmentation or decibel gradation) according to uniform linear intervals or proportional logarithmic intervals to adapt to the dynamic range of acoustic signal amplitude.

[0176] Next, we perform counting and statistics. That is, based on the two-dimensional data pairs of each pulse, we count the number of pulses contained in each phase-amplitude unit. In other words, we count the number of pulses contained in each combination of phase interval × amplitude interval, thus forming a three-dimensional matrix. For example, assuming that a total of 6 pulses are detected in the 54° phase interval × 150mV amplitude interval, it means that the number of pulses contained in this phase-amplitude unit is 6, and we count 6 for this phase-amplitude unit.

[0177] Finally, the PRPD spectrum is plotted. With phase as the x-axis and pulse amplitude as the y-axis, and based on the statistical number of pulses, each phase-amplitude unit is visualized to obtain the PRPD spectrum.

[0178] In the PRPD graph, multiple phase intervals are evenly distributed on the horizontal axis, and multiple amplitude intervals are evenly distributed on the vertical axis. In other words, for the horizontal axis, the phase angle (0°-360°) is evenly distributed according to the divided phase intervals. For the vertical axis, the pulse amplitude (units such as mV, dB, or pC) is evenly distributed according to the divided amplitude intervals. For the third dimension (visual dimension), the count can be represented by color intensity or dot size during visualization. The more counts a phase-amplitude unit corresponds to, the darker its color (e.g., using a gradient from blue to red) or the larger its dot, thus visually representing the concentration of the pulse.

[0179] 1202-2. Generate a thermal image based on the thermal imaging signal, wherein the thermal image is used to describe the temperature distribution on the surface of the device under test in a visual manner.

[0180] In this embodiment of the application, the generation of the thermal image includes the following steps.

[0181] 1. Infrared radiation acquisition: The infrared lens and infrared detector of the thermal imaging module work together to capture the signal.

[0182] Infrared lens: Filters out visible light and interference bands, allowing only infrared radiation from the device under test to enter the module, thus achieving accurate screening of effective radiation signals.

[0183] Infrared detector: The core uses a focal plane array, which consists of thousands of miniature infrared sensing units (such as microbolometers). Each sensing unit corresponds to a pixel on the surface of the device being measured. It can absorb infrared radiation and generate physical changes such as resistance and voltage, converting infrared radiation energy into a weak electrical signal.

[0184] 2. Signal processing: Convert electrical signals into temperature data.

[0185] Signal amplification and noise reduction: For the weak electrical signal output by the detector, a dedicated circuit is used to amplify the signal and filter out environmental noise such as background radiation interference to improve signal quality.

[0186] Temperature calibration calculation: Combining the detector response curve with the emissivity of the device under test (characterizing the device's infrared radiation capability, supporting preset or automatic compensation), the algorithm converts the radiation energy corresponding to the electrical signal into the actual temperature value of each pixel, in °C or °F.

[0187] 3. Thermal mapping visualization: Enables the conversion of temperature data into thermal imaging maps.

[0188] Pseudo-color mapping: A thermal color table is used to complete a one-to-one mapping between temperature values ​​and colors, and corresponding characteristic colors are assigned to different temperature ranges. For example, low temperature areas are presented as blue and green, medium temperature areas as yellow and orange, and high temperature areas as red and white.

[0189] Pixel stitching and imaging: Following the arrangement of the detector array, the color information of all pixels is stitched together to generate a complete two-dimensional thermal image (thermal mapping), intuitively presenting the temperature distribution characteristics of the surface of the device under test. Color differences in the image directly correspond to temperature differences, achieving a visualization effect of temperature indication by color.

[0190] 1202-3. Generate a sound pressure distribution image based on acoustic signals, wherein the sound pressure distribution image is used to describe the sound pressure intensity at each spatial location in the target scene; based on the mapping relationship between sound pressure intensity and color, visualize the sound pressure distribution image to obtain an acoustic imaging map covering the target scene.

[0191] This step is implemented by the imager based on beamforming and other acoustic imaging algorithms.

[0192] In the acoustic imaging process, the imager first constructs a sound pressure distribution image based on acoustic signals. This image accurately represents the sound pressure intensity at each spatial location within the target scene, fully showcasing the spatial distribution and intensity differences of the sound field. Next, the imager visualizes the sound pressure distribution image based on a pre-defined sound pressure intensity and color mapping relationship. This involves matching different sound pressure intensity values ​​with corresponding colors, ultimately generating an acoustic image that visually represents the sound field characteristics and completely covers the scene being detected, thus achieving a visual representation of the sound field distribution.

[0193] It should be noted that, in addition to PRPD maps, acoustic imaging maps, and thermal imaging maps, the embodiments of this application can also generate acoustic-thermal fusion results, as detailed in steps 1202-4 below.

[0194] 1202-4. Extract the thermal image features from the thermal imaging image and the acoustic image features from the acoustic imaging image; perform feature fusion on the thermal image features and acoustic image features to obtain multimodal features; perform visualization drawing based on the multimodal features to obtain the acoustic-thermal anomaly correlation map.

[0195] In this embodiment, the acoustic-thermal anomaly correlation diagram is used to describe the degree of correlation between acoustic anomalies and thermal anomalies at various locations of the device under test. Acoustic anomalies refer to equipment malfunctions identified through acoustic signals, while thermal anomalies refer to equipment malfunctions identified through thermal imaging signals.

[0196] In some embodiments, the imager first extracts acoustic and thermal features based on a two-stream convolutional-attention network. The two-stream convolutional-attention network employs a parallel processing structure, enabling the extraction of data features from both modalities. Specifically, the network includes two parallel convolutional branches: an acoustic convolutional branch specifically processes the acoustic image, responsible for extracting acoustic features (such as the location and amplitude of sound pressure anomalies), and a thermal convolutional branch specifically processes the thermal image, responsible for extracting thermal features (such as the location, temperature rise amplitude, and hotspot features of temperature anomalies).

[0197] It should be noted that, in addition to inputting the acoustic image into the acoustic convolution branch for acoustic image feature extraction, the embodiments of this application also support using the time domain features or frequency domain features of the acoustic signal as model input, and this application does not limit this.

[0198] After feature extraction, the cross-modal attention layer of the dual-stream convolutional-attention network performs feature alignment and fusion. Feature alignment refers to the cross-modal attention layer automatically calculating the spatial correlation between acoustic and thermal features (e.g., whether the coordinates of acoustic anomalies identified by the acoustic convolutional branch match the coordinates of thermal anomalies identified by the thermal convolutional branch) and the temporal correlation (e.g., whether acoustic anomalies and thermal anomalies occur synchronously). This ensures that the two types of features are accurately matched in location and time, avoiding misaligned fusion where an acoustic anomaly is at location A and a thermal anomaly is at location B.

[0199] After feature alignment, the cross-modal attention layer assigns higher weights to highly correlated features and lower weights to low-correlation or interfering features. Then, a fully connected layer integrates the weighted acoustic and thermal features into acoustic-thermal fusion features. The fused features are mapped back to a spatial coordinate system, ultimately presenting the correlation between acoustic and thermal anomalies at each spatial location in the form of a heatmap. Higher correlation (e.g., partial discharge simultaneously causing acoustic and thermal anomalies) results in a more prominent color in the heatmap (e.g., a redder or brighter color), indicating a higher probability that the location simultaneously satisfies both acoustic and thermal anomalies, thus representing a high-confidence fault point; lower correlation (only acoustic or only thermal anomalies) results in a lighter color.

[0200] In summary, the embodiments of this application implement an AI-driven acoustic-thermal deep fusion algorithm. This algorithm extracts acoustic and thermal image features from two convolutional branches of a dual-stream convolutional attention network, and then achieves feature alignment and fusion through the network's cross-modal attention layer, ultimately generating an acoustic-thermal anomaly correlation map. Furthermore, this algorithm can output acoustic-thermal fusion results in real time at high frame rates above 30Hz, significantly improving fault detection accuracy in multiple scenarios such as partial discharge, gas-liquid leakage, and mechanical friction.

[0201] 1203. The imager identifies the operating status of the device under test based on acoustic signals and obtains the operating status identification result; wherein, at least one generated image and the operating status identification result are used to characterize the current operating status of the device under test from different perspectives.

[0202] In this embodiment, regardless of the type of device under test, the imager can identify the operating status of the device based on acoustic signals. Taking power equipment as an example, while generating a PRPD map, the imager can also identify the operating status of the power equipment based on acoustic signals to determine whether partial discharge exists. This embodiment supports both rule-based threshold judgment and machine learning-based intelligent recognition, which will be described below.

[0203] I. Rule-based threshold judgment.

[0204] For this identification method, firstly, an acoustic feature set for identifying the operating status of the device under test is determined, and then an anomaly judgment threshold for each acoustic feature in the acoustic feature set is obtained. In some embodiments, corresponding threshold ranges can be set for features such as peak value, kurtosis, and spectral peak value included in the acoustic feature set according to industry standards or historical operation and maintenance data (for example, setting the characteristic frequency band of kurtosis > 5 and spectral peak value to 150-300kHz). This application does not limit this.

[0205] Next, a multi-feature joint judgment is performed. That is, the feature values ​​of the acoustic signal (collecting multiple pulses) are obtained in each feature dimension, i.e., the feature value of the acoustic signal corresponding to each acoustic feature in the acoustic feature set is calculated. If the obtained feature values ​​meet the threshold conditions corresponding to a preset number of acoustic features, it is determined that the device under test has an operational abnormality. In other words, if the acoustic signal simultaneously meets a preset number of threshold conditions (such as peak value meeting the standard, kurtosis meeting the standard, synchronization with voltage period, etc.), it is determined that the device under test has an operational abnormality; if only a single feature or a few features meet the threshold conditions, it is highly likely that it is caused by interference signals.

[0206] II. Intelligent recognition based on machine learning.

[0207] For this type of identification, the imager calls a machine learning model that matches the device type of the device under test and performs the step of identifying the operating status of the device under test based on acoustic signals.

[0208] Taking the tested device as an electrical device as an example, the above machine learning model is a partial discharge identification model. In some embodiments, during the training process of this model, a sample dataset is first constructed, which includes a large number of labeled partial discharge signal samples and interference signal samples.

[0209] The training objective is to construct a machine learning model capable of accurately distinguishing between acoustic signals of partial discharge in power equipment and acoustic signals of environmental interference. The core logic is to allow the model to learn the mapping relationship between partial discharge signal features and equipment operating states from labeled sample data. In this embodiment, the model is either a binary classification model or a multi-class classification model. The binary classification model is used to determine whether partial discharge exists in the power equipment, while the multi-class classification model can not only determine whether partial discharge exists but also further distinguish the type of partial discharge. Furthermore, the model input consists of acoustic features, such as time-domain, frequency-domain, or time-frequency domain features of the acoustic signal; this application does not limit the specific features. The model output includes, but is not limited to, the category label and confidence level of the equipment operating state.

[0210] Subsequently, based on this sample dataset, the network structure responsible for identifying the operating state in the power partial discharge identification model was trained.

[0211] Since the raw acoustic signal contains a large amount of redundant information and noise, the sample dataset needs to be preprocessed to improve data quality. Next, feature extraction is performed on the sample data included in the sample dataset to obtain sample feature vectors. The extracted sample feature vectors can be further divided into multiple subsets, such as training and testing sets, to meet the needs of training and testing. Regarding the choice of network structure, deep learning architectures, such as convolutional neural networks or recurrent neural networks, are typically chosen; this application does not impose any restrictions on this.

[0212] During the training phase, the network parameters are first initialized. Then, the training set is input into the network structure responsible for identifying the running state. This network structure calculates the predicted labels using a forward propagation algorithm and calculates the loss value by comparing the error between the predicted and true labels. Subsequently, the network structure further updates the network parameters using a backpropagation algorithm to gradually reduce the calculated loss value until a preset training termination condition is met. The training termination condition can be either reaching a preset number of iterations or the loss value falling below a preset threshold; this application does not specify either condition.

[0213] During the validation phase, the trained network structure is evaluated using a test set, for example, by measuring accuracy, precision, recall, or F1 score. If the performance of the trained network structure meets the preset metrics, the network parameters, network structure, feature preprocessing rules, and other information are encapsulated and saved to generate a deployable model file for use in the subsequent model inference phase.

[0214] During the model inference phase, after acoustic signals are acquired and features are extracted in the target scene, the extracted acoustic features are input into the model, which then outputs the corresponding operational status identification result. This operational status identification result includes, but is not limited to, the operational status type of the device under test (e.g., partial discharge), the confidence level of the prediction result, etc., which are not limited in this application.

[0215] When the device under test is a pressure vessel or pressure pipeline, the same processing method as above is used to train the gas and liquid leakage identification model and apply it; when the device under test is a mechanical structure, the same processing method as above is used to train the structural sound anomaly identification model and apply it, which will not be repeated here.

[0216] 1204. The imager outputs the operating status recognition results and displays them visually based on at least one generated image.

[0217] In the embodiments of this application, the running status recognition result can be output in the form of voice broadcast or displayed on the screen in the form of text. This application does not limit this.

[0218] In addition, the imager can be used for visualization, including but not limited to: split-screen display on a display screen, layered overlay display on the same screen, or multimodal fusion display (such as displaying the results of sound and heat fusion). The following example illustrates visualization based on at least one generated image.

[0219] Example 1: Using a real-world image as the base image, overlay an acoustic imaging image onto the real-world image.

[0220] In this display method, the acoustic imaging image is located on the layer above the visible light image, thus achieving a visualization effect with the visible light image as the background and the acoustic imaging image as the foreground. Furthermore, during the overlay display, spatial registration is performed on the real-world image and the acoustic imaging image to ensure precise spatial matching and no misalignment.

[0221] In this example, spatial registration is used to precisely align the spatial location and coordinate system of the visible light image and the acoustic image. The purpose is to ensure that the acoustic anomaly area in the acoustic image can accurately correspond to the actual physical location of the device in the visible light image, avoiding misalignments such as "the acoustic anomaly is shown at point A on the image, but the actual device failure is located at point B".

[0222] The following section provides a detailed introduction to spatial registration.

[0223] Spatial registration is a technique that uses factory calibration parameters to solve for the transformation matrix between the coordinate systems corresponding to different modal data and a unified reference coordinate system, thereby achieving spatial alignment of multimodal data. In other words, establishing spatial mapping relationships is the process of unifying visible light image coordinates (pixel coordinates in visible light images, from image space), thermal imaging coordinates (pixel coordinates in thermal images, from thermal imaging space), and acoustic spatial coordinates (sound source location coordinates, from detection space) to the same reference coordinate system based on factory calibration parameters.

[0224] In this system, the visible light image coordinates and thermal imaging coordinates correspond to a two-dimensional pixel coordinate system, while the acoustic space coordinates correspond to a three-dimensional detection coordinate system. In some embodiments, the unified reference coordinate system can be either a three-dimensional detection coordinate system or a world coordinate system (a physical coordinate system with the key components of the device under test as the origin), allowing all modal data to be mapped to this coordinate system. This application does not limit this choice. In some embodiments, taking the unified reference coordinate system as the three-dimensional detection coordinate system as an example, the above-mentioned spatial registration operation between the real-scene image and the acoustic imaging image refers to: based on factory calibration parameters, transforming the two-dimensional coordinates of any pixel in the visible light image to the three-dimensional detection coordinate system to obtain the corresponding three-dimensional detection space coordinates of that pixel.

[0225] Example 2: Simultaneously display PRPD maps and thermal images.

[0226] In this example, synchronous display means that the PRPD map and the thermal image are presented in conjunction on the same interface and in the same time dimension, and the two images represent the operating data of the device under test in the same detection period and the same spatial area (the partial discharge location corresponds to the temperature anomaly location).

[0227] This display method uses a thermal image as the base map and overlays a PRPD map on it to help operators intuitively identify the temperature anomaly area corresponding to the partial discharge location, thus realizing the linkage visualization of discharge characteristics and temperature rise characteristics.

[0228] Example 3: Simultaneously display acoustic and thermal images.

[0229] This display method achieves a visual imaging effect of sound and heat on the same screen by performing spatial registration on the acoustic imaging image and the thermal image. In this example, spatial registration is used to map the location of the acoustic anomaly represented in the acoustic imaging image and the location of the temperature anomaly represented in the thermal imaging image to the same physical location of the device under test, thus achieving precise spatial alignment of the acoustic and thermal anomaly features.

[0230] Example 4: Display a fused image of a real-world scene, an acoustic image, and a thermal image; use the fused image as a base image and overlay a PRPD map on top of it.

[0231] In this display method, the fused image refers to an integrated image formed by precisely fusing multiple layers of visible light images, acoustic images, and thermal images through spatial registration technology. Using a real-scene image as a base, the sound field distribution characteristics of the acoustic image and the temperature distribution characteristics of the thermal image are simultaneously superimposed, achieving a fused presentation of the actual appearance of the device, areas of acoustic anomalies, and areas of thermal anomalies in the same image, with precise spatial correspondence and no deviation or misalignment among the three.

[0232] Example 5: Display the correlation diagram of acoustic and thermal anomalies.

[0233] Unlike Example 3, this display method does not involve layered overlay. Instead, it deeply integrates thermal and acoustic features to generate an integrated image. This means that the distribution characteristics of acoustic and thermal anomalies at various spatial locations of the tested device are presented synchronously in a single image, without the need for multiple layers of images. It can intuitively reflect the correlation strength between acoustic and thermal anomalies.

[0234] Figure 14 This is a schematic diagram illustrating multimodal visualization using an imager provided in an embodiment of this application. For example... Figure 14As shown, when the imager performs visualization, it uses the thermal image as a base map and draws a PRPD map in real time on top of the thermal image layer. This allows operators to view both the PRPD map and the thermal image simultaneously. Furthermore, the PRPD map has a certain degree of transparency to avoid obscuring the content of the thermal image and ensure clear visibility of temperature anomalies.

[0235] Figure 15 This is a schematic diagram illustrating multimodal visualization using another imager provided in an embodiment of this application. For example... Figure 15 As shown, the imager displays a split-screen view when providing visualization. The left side of the screen displays a visible light image, showing the physical appearance of the device under test. The right side displays a thermal image, using a color gradient from red to yellow to blue to represent temperature levels (red for high temperatures and blue for low temperatures). Additionally, the right side also displays a sound pressure level scale to indicate the strength of the acoustic signal. Figure 15 In the image, the PRPD map is a layered overlay of visible light and thermal images with a certain degree of transparency. Additionally, the display shows "Surface Discharge: 120.00%", indicating that the device under test is currently identified by the imager as having a surface discharge type partial discharge fault, and the confidence level of this prediction is 100%.

[0236] In summary, the embodiments of this application achieve automatic identification of the operating status of the device under test (DUT) based on signals collected in the target scene that reflect the DUT's operating status. Since no manual identification of the DUT's operating status is required, compared to manual inspection methods, this not only saves labor costs but also increases efficiency. Abnormalities in the DUT can be detected and addressed promptly, significantly improving the reliability and stability of the equipment's operation.

[0237] In addition to outputting the operational status identification results, this solution also provides a visual display based on at least one generated image. The generated image and the operational status identification results are used to characterize the current operational status of the device under test from different perspectives. This visualization method allows operators to intuitively view the acoustic, thermal, and discharge anomaly characteristics of the equipment, helping them efficiently grasp the equipment's operational status. Furthermore, this solution, with its automated detection and judgment process, improves the accuracy of equipment operational status identification and is not affected by subjective human factors. Moreover, because this solution adopts a non-contact detection mode, it eliminates the need for operators to perform anomaly detection on the device under test at close range, thus significantly reducing on-site operational risks and fully ensuring the personal safety of operators.

[0238] It should be noted that after outputting the operational status identification results of the tested device and displaying them visually, the imager can also use the analysis and interaction module to provide a natural language explanation of the mechanism, risk level description, and generate processing and retesting suggestions for the given anomaly detection results. This part will be described in detail below.

[0239] 1. Structured packaging.

[0240] This step is used to generate a structured inspection record. In some embodiments, this structured inspection record is obtained by encapsulating information of at least one of the following: A. The first scene information of the target scene, such as the device type and the scene category of the scene to be detected, is not limited in this application.

[0241] B. Candidate detection area of ​​the device under test.

[0242] C. Thermal anomaly data used to reflect the surface temperature of the tested equipment, such as the highest temperature and temperature rise, etc. This application does not limit this.

[0243] D. Acoustic characteristics of acoustic signals, such as peak value, kurtosis, spectral peak value, characteristic frequency, etc., are not limited in this application.

[0244] E. Parameter information of the PRPD spectrum, such as phase distribution, amplitude distribution, number of pulses included in each phase-amplitude unit, etc., are not limited in this application.

[0245] F. The results of identifying the operating status of the device under test.

[0246] It should be noted that the information to be encapsulated may include more or less information than the examples above, and this application does not limit this.

[0247] 2. Constructing the prompt message.

[0248] This step is used to construct a prompt text based on the structured detection record and industry knowledge. The prompt text includes at least one of the following: A. Second scene information of the target scene; wherein the second scene information is more information-rich than the first scene information. In some embodiments, the second scene information includes, but is not limited to, device type, scene category of the scene to be detected, detection purpose, etc., which are not limited in this application.

[0249] B. Output constraint information, such as target language, target audience (operations engineers / managers / end customers), report style, etc., which are not limited in this application.

[0250] C. Anomaly evaluation information of the tested equipment, such as the degree of abnormality in equipment operation, risk level, and trend of change (which can be compared with historical data). This application does not limit this.

[0251] 3. Natural language text generation.

[0252] This step uses the prompt text as input to the large language model, and then uses the large language model to generate the target text in natural language form.

[0253] In some embodiments, the target text includes at least one of the following: an overview of the current operational status identification, explanatory information on the operational status identification results, explanatory information on the risk level, and processing and retesting recommendations.

[0254] The above overview includes, but is not limited to, the object, time, location, or main findings of this test, which this application does not limit. The explanatory information above is used to explain the possible mechanisms of suspected partial discharge, gas / liquid leakage, or mechanical structural failure. The explanatory information above is used to explain the safety and equipment operation risks corresponding to the technical assessment results in layman's terms. The handling and retesting recommendations are used to provide suggestions such as shutdown / reduction of load, maintenance, enhanced monitoring, or retesting cycles, which this application does not limit.

[0255] 4. Template adaptation in multiple scenarios.

[0256] Since the imager can be applied to various detection scenarios, this embodiment also uses prompt text as input to a large language model. The large language model then generates target text that conforms to the text output framework. This text output framework matches the scenario to be detected. In other words, during the generation of natural language text, the imager's analysis and interaction module automatically switches between different semantic templates based on the scenario to be detected. For partial discharge scenarios, the text may focus on explaining the discharge type and its impact on the equipment's insulation life; for gas / liquid leakage scenarios in pressure vessels, the text may focus on the leakage location, leakage level, and its impact on safety and environmental protection; for abnormal mechanical structure noise scenarios, the text may focus on possible bearing or gear failures and their impact on equipment operational reliability.

[0257] 5. Interactive question and answer in natural language format.

[0258] In this embodiment, the operator can ask questions in natural language to the imager via touch input or voice input. Correspondingly, the imager's analysis and interaction module acquires the user's question, inputs it along with the aforementioned structured detection record into the large language model, obtains the response from the large language model, and finally outputs the response to the user's question.

[0259] In some embodiments, user questions may be such as "Does this test result require immediate shutdown of the equipment?", "What is the approximate leakage level?", or "Can the equipment continue to operate for a week if there are abnormal mechanical noises?", etc., and this application does not limit them.

[0260] In addition, the imager can display the above-mentioned response content on the screen or automatically broadcast the above-mentioned response content via voice. This application does not limit this.

[0261] 6. Automatic report generation.

[0262] In some embodiments, the imager's display screen is equipped with a report generation button. In response to triggering the report generation button, the imager automatically calls the report generation module to generate and output a tamper-proof detection report (also known as a device status report) based on a preset report template.

[0263] This report generation method requires no manual typesetting and takes very little time, which can greatly improve the timeliness and consistency of on-site testing reports.

[0264] It should be noted that the report generation button can also be a physical button located on the imager; this application does not limit this. Furthermore, the test report supports both online and offline generation methods; this application also does not limit this.

[0265] In some embodiments, the device status report includes at least one of the following. That is, the device status report is generated by filling in at least one of the following into a preset report template.

[0266] A. The above-mentioned generated at least one image, such as the superimposed display result of PRPD map and thermal image, or the fusion of acoustic image, thermal image and visible light image, are not limited in this application.

[0267] B. Sound source location coordinates, where the sound source location coordinates are used to indicate the position of the device under test in the scene to be tested.

[0268] B. Temperature trend curve, which describes the trend of surface temperature change of the tested equipment over time.

[0269] C. The structured detection records generated above.

[0270] D. The target text in natural language form output by the large language model. That is, the text in natural language form generated by the large language model can be directly used as the main content of the report.

[0271] It should be noted that equipment status reports may include more or less information than the examples above, and this application does not limit this. For example, temperature distribution maps and operator comments may also be included in the report.

[0272] In summary, the imager-based device operation status recognition method provided in this application embodiment realizes a complete end-to-end signal processing flow, from data acquisition, device anomaly detection and visualization, result interpretation, and automatic report generation. Operators can generate and output detection reports on-site with a single click, offering greater portability, flexibility, and intelligence. It is applicable to multiple scenarios, possesses multi-scenario intelligent detection capabilities, and allows operators to make appropriate decisions on-site.

[0273] Figure 16 This is a schematic diagram of the structure of a device for identifying the operating status provided in an embodiment of this application. See also... Figure 16 The device includes: The first acquisition module 1601 is configured to acquire acoustic signals collected in the target scene; The first generation module 1602 is configured to generate at least one image based on the acoustic signal, using the device type of the device under test in the target scene as a constraint. The identification module 1603 is configured to identify the operating status of the device under test based on the acoustic signal, and obtain an operating status identification result; wherein, the at least one image and the operating status identification result are used to characterize the current operating status of the device under test from different perspectives; The output module 1604 is configured to output the running status recognition result and visualize it based on the at least one image.

[0274] This application embodiment achieves automatic identification of the operating status of the device under test (DUT) based on signals collected in the target scene that reflect the DUT's operating status. Since no manual operation is required, compared to manual inspection methods, this not only saves labor costs but also increases efficiency. Abnormalities in the DUT can be detected and addressed promptly, significantly improving the reliability and stability of the equipment. Furthermore, in addition to outputting the operating status identification results, this solution also provides a visual display based on at least one generated image. The generated image and the operating status identification results are used to characterize the current operating status of the DUT from different perspectives. This visualization allows operators to intuitively view the acoustic, thermal, and discharge anomalies of the equipment, helping them efficiently grasp the equipment's operating status. Moreover, this solution, with its automated detection and judgment process, improves the accuracy of equipment operating status identification and is not affected by subjective human factors. Additionally, because this solution uses a non-contact detection mode, it eliminates the need for operators to perform close-range anomaly detection on the DUT, significantly reducing on-site operational risks and fully ensuring the personal safety of operators.

[0275] In some embodiments, the first acquisition module is further configured to acquire the thermal imaging signal radiated by the device under test; The first generation module is configured to generate the at least one image based on one or more of the acoustic signal or the thermal imaging signal, using the device type as a constraint.

[0276] In other embodiments, when the device under test is an electrical device, the first generation module is configured to: Based on the acoustic signal, a PRPD spectrum is generated in real time. The PRPD spectrum is used to describe the partial discharge characteristics of the device under test through phase distribution. A thermal imaging image is generated based on the thermal imaging signal, and the thermal imaging image is used to describe the temperature distribution on the surface of the device under test in a visual manner. A sound pressure distribution image is generated based on the acoustic signal. The sound pressure distribution image is used to describe the sound pressure intensity at each spatial location in the target scene. Based on the mapping relationship between sound pressure intensity and color, the sound pressure distribution image is visualized and drawn to obtain an acoustic imaging map covering the target scene. The thermal features of the thermal imaging image and the acoustic features of the acoustic imaging image are extracted; the thermal features and the acoustic features are fused to obtain multimodal features; the multimodal features are visualized to obtain an acoustic-thermal anomaly correlation map; wherein, the acoustic-thermal anomaly correlation map is used to describe the degree of correlation between acoustic anomalies and thermal anomalies at various locations of the tested device; the acoustic anomaly refers to the abnormal operation of the device identified by the acoustic signal; the thermal anomaly refers to the abnormal operation of the device identified by the thermal imaging signal.

[0277] In other embodiments, the first generation module is configured to: Feature extraction is performed on the acoustic signal to obtain an effective pulse signal; the PRPD map is generated based on the effective pulse signal; wherein, the feature extraction includes at least one of the following: Temporal feature extraction is performed on the preprocessed acoustic signal; Frequency domain features are extracted from the preprocessed acoustic signal; Time-frequency domain features are extracted from the preprocessed acoustic signal.

[0278] In other embodiments, the first generation module is configured to: Obtain the amplitude and trigger timestamp of each pulse included in the valid pulse signal; Acquire phase data of the power frequency voltage signal that is synchronously acquired with the acoustic signal; the phase data includes the phase of the power frequency voltage signal at each moment; Using the zero-crossing point of the power frequency voltage signal as the zero phase, a mapping relationship between time and phase is established; Based on the trigger timestamp of each pulse and the mapping relationship, the phase of each pulse is determined, and a two-dimensional data pair of each pulse is obtained; wherein, for any pulse, the two-dimensional data pair of the pulse includes the amplitude and phase of the pulse; The PRPD map is generated based on the two-dimensional data pairs of each pulse.

[0279] In other embodiments, the first generation module is configured to: The phase is divided into multiple phase intervals according to a preset step size; The pulse amplitude is divided into multiple amplitude intervals according to linear or logarithmic intervals; Based on the two-dimensional data pairs of each pulse, the number of pulses contained in each phase-amplitude unit is counted; With phase as the horizontal axis and pulse amplitude as the vertical axis, each phase-amplitude unit is visualized based on the statistical number of pulses to obtain the PRPD spectrum; In the PRPD spectrum, the multiple phase intervals are uniformly distributed on the horizontal axis, and the multiple amplitude intervals are uniformly distributed on the vertical axis.

[0280] In other embodiments, the output module is configured as follows: Obtain a real-world image of the target scene, and use the real-world image as a base image to overlay and display the acoustic imaging image; or, Simultaneously display the PRPD map and the thermal image; or, Simultaneously display the acoustic image and the thermal image; or, Display a fused image of the real-world image, the acoustic image, and the thermal image; use the fused image as a base image and overlay the PRPD map onto the fused image; or, The correlation diagram of the aforementioned acoustic and thermal anomalies is shown.

[0281] In other embodiments, the device further includes: The first acquisition module is further configured to acquire a real-world image of the target scene; The processing module is configured to perform target detection on the real-scene image and obtain target detection results; the target detection results include the device type and location coordinates of the device under test; The processing module is further configured to perform thermal anomaly detection based on the thermal image to obtain thermal anomaly detection results; the thermal anomaly detection results include the contour coordinates of the detected thermal anomaly region; The processing module is further configured to determine the candidate detection area of ​​the device under test based on the target detection result and the thermal anomaly detection result; The output module is further configured to mark the candidate detection area on the real-world image and the thermal image, and output prompt information; the prompt information is used to prompt the imager to be aligned with the candidate detection area.

[0282] In other embodiments, the processing module is further configured to: Based on coordinate transformation rules, the pixel coordinates of the candidate detection region in the image space are converted into the acoustic coordinates or beam pointing parameters of the acoustic signal acquisition module of the imager in the detection space.

[0283] In other embodiments, the identification module is configured as follows: Determine a set of acoustic features used to identify the operating status of the device under test; obtain an anomaly threshold for each acoustic feature in the set; obtain the feature value of each acoustic feature; if the obtained feature value satisfies the threshold conditions corresponding to a preset number of acoustic features, determine that the device under test has an operating anomaly; or, Invoke a machine learning model that matches the device type and perform the step of identifying the operating status of the device under test based on the acoustic signal.

[0284] In other embodiments, the device further includes: The second generation module is configured to generate structured detection records; the structured detection records are obtained by encapsulating at least one of the following: first scene information of the target scene, candidate detection area of ​​the device under test, thermal anomaly data reflecting the surface temperature of the device under test, acoustic features of the acoustic signal, parameter information of the PRPD spectrum, or the operating status identification result. The construction module is configured to construct prompt text based on the structured detection records and industry knowledge; the prompt text includes at least one of the following: second scene information of the target scene, output constraint information, or abnormal evaluation information of the tested device; the second scene information has a greater amount of information than the first scene information; The third generation module is configured to use the prompt text as input to the large language model and generate target text through the large language model; wherein the target text includes at least one of the following: an overview of the current operation status identification, explanatory information of the operation status identification results, explanatory information on the risk level, and processing and retesting suggestions.

[0285] In other embodiments, the third generation module is configured as follows: Using the prompt text as input to the large language model, the large language model generates target text that conforms to the text output framework. The text output framework is matched with the target scene.

[0286] In other embodiments, the device further includes: The second acquisition module is configured to acquire user questions; The input module is configured to input the user's question and the structured detection record into the large language model. The third acquisition module is configured to acquire the response content fed back by the large language model; The output module is also configured to output the response content.

[0287] In other embodiments, a report generation button is displayed on the screen; the output module is further configured to output a device status report in response to a triggering operation of the report generation button; wherein the device status report includes at least one of the following: The at least one image; Sound source location coordinates, which are used to indicate the position of the device under test in the target scene; Temperature trend curve, which is used to describe the change trend of the surface temperature of the device under test over time; The structured detection record; The target text.

[0288] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0289] It should be noted that the device operation status identification device provided in the above embodiments is only illustrated by the division of the above functional modules when identifying the device operation status. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device operation status identification device and the device operation status identification method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0290] In some embodiments, this application also provides a device operating status identification component. The component includes a processor and a memory. The memory stores computer program code, which is loaded and executed by the processor to implement the device operating status identification method described above.

[0291] In some embodiments, this application also provides a computer-readable storage medium, such as a memory including computer program code, which can be executed by the processor of a device operating state identification component to implement the device operating state identification method described above. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, or optical data storage device, etc.

[0292] In some embodiments, this application also provides a computer program product, which includes computer program code stored in a computer-readable storage medium. A processor reads the computer program code from the computer-readable storage medium to execute the computer program code, causing the imager to perform the device operating status identification method described above.

[0293] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0294] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for identifying the operating status of equipment, characterized in that, The method includes: Acquire acoustic signals collected in the target scene; Using the device type of the device under test in the target scene as a constraint, at least one image is generated based on the acoustic signal; Based on the acoustic signal, the operating status of the device under test is identified to obtain an operating status identification result; wherein, the at least one image and the operating status identification result are used to characterize the current operating status of the device under test from different perspectives; The operating status identification result is output and visualized based on the at least one image.

2. The method according to claim 1, characterized in that, The method further includes: Acquire the thermal imaging signal radiated by the device under test; The generation of at least one image based on the acoustic signal includes: Using the device type as a constraint, the at least one image is generated based on one or more of the acoustic signal or the thermal imaging signal.

3. The method according to claim 2, characterized in that, When the device under test is an electrical device, generating the at least one image based on one or more of the acoustic signal or the thermal imaging signal includes: Based on the acoustic signal, a partial discharge phase distribution (PRPD) map is generated in real time. The PRPD map is used to describe the partial discharge characteristics of the device under test through the phase distribution. A thermal imaging image is generated based on the thermal imaging signal, and the thermal imaging image is used to describe the temperature distribution on the surface of the device under test in a visual manner. A sound pressure distribution image is generated based on the acoustic signal. The sound pressure distribution image is used to describe the sound pressure intensity at each spatial location in the target scene. Based on the mapping relationship between sound pressure intensity and color, the sound pressure distribution image is visualized and drawn to obtain an acoustic imaging map covering the target scene. The thermal features of the thermal imaging image and the acoustic features of the acoustic imaging image are extracted; the thermal features and the acoustic features are fused to obtain multimodal features; the multimodal features are visualized to obtain an acoustic-thermal anomaly correlation map; wherein, the acoustic-thermal anomaly correlation map is used to describe the degree of correlation between acoustic anomalies and thermal anomalies at various locations of the tested device; the acoustic anomaly refers to the abnormal operation of the device identified by the acoustic signal; the thermal anomaly refers to the abnormal operation of the device identified by the thermal imaging signal.

4. The method according to claim 3, characterized in that, The generation process of the PRPD map includes: Feature extraction is performed on the acoustic signal to obtain an effective pulse signal; the PRPD map is generated based on the effective pulse signal; wherein, the feature extraction includes at least one of the following: Temporal feature extraction is performed on the preprocessed acoustic signal; Frequency domain features are extracted from the preprocessed acoustic signal; Time-frequency domain features are extracted from the preprocessed acoustic signal.

5. The method according to claim 4, characterized in that, The process of generating the PRPD map based on the effective pulse signal includes: Obtain the amplitude and trigger timestamp of each pulse included in the valid pulse signal; Acquire phase data of the power frequency voltage signal that is synchronously acquired with the acoustic signal; the phase data includes the phase of the power frequency voltage signal at each moment; Using the zero-crossing point of the power frequency voltage signal as the zero phase, a mapping relationship between time and phase is established; Based on the trigger timestamp of each pulse and the mapping relationship, the phase of each pulse is determined, and a two-dimensional data pair of each pulse is obtained; wherein, for any pulse, the two-dimensional data pair of the pulse includes the amplitude and phase of the pulse; The PRPD map is generated based on the two-dimensional data pairs of each pulse.

6. The method according to claim 5, characterized in that, The generation of the PRPD map based on the two-dimensional data pairs of each pulse includes: The phase is divided into multiple phase intervals according to a preset step size; The pulse amplitude is divided into multiple amplitude intervals according to linear or logarithmic intervals; Based on the two-dimensional data pairs of each pulse, the number of pulses contained in each phase-amplitude unit is counted; With phase as the horizontal axis and pulse amplitude as the vertical axis, each phase-amplitude unit is visualized based on the statistical number of pulses to obtain the PRPD spectrum; In the PRPD spectrum, the multiple phase intervals are uniformly distributed on the horizontal axis, and the multiple amplitude intervals are uniformly distributed on the vertical axis.

7. The method according to claim 3, characterized in that, The visualization based on the at least one image includes: Obtain a real-world image of the target scene, and use the real-world image as a base image to overlay and display the acoustic imaging image; or, Simultaneously display the PRPD map and the thermal image; or, Simultaneously display the acoustic image and the thermal image; or, Display a fused image of the real-world image, the acoustic image, and the thermal image; use the fused image as a base image and overlay the PRPD map onto the fused image; or, The correlation diagram of the aforementioned acoustic and thermal anomalies is shown.

8. The method according to claim 3, characterized in that, The method further includes: A real-world image of the target scene is acquired, and target detection is performed on the real-world image to obtain a target detection result; the target detection result includes the device type and location coordinates of the device under test; Thermal anomaly detection is performed based on the thermal image to obtain thermal anomaly detection results; the thermal anomaly detection results include the contour coordinates of the detected thermal anomaly region. Based on the target detection results and the thermal anomaly detection results, the candidate detection area of ​​the device under test is determined; The candidate detection area is marked on the real-world image and the thermal image, and a prompt message is output; the prompt message is used to prompt the imager to be aligned with the candidate detection area.

9. The method according to claim 8, characterized in that, The method further includes: Based on coordinate transformation rules, the pixel coordinates of the candidate detection region in the image space are converted into the acoustic coordinates or beam pointing parameters of the acoustic signal acquisition module of the imager in the detection space.

10. The method according to claim 1, characterized in that, The step of identifying the operating status of the device under test based on the acoustic signal includes: Determine a set of acoustic features used to identify the operating status of the device under test; obtain an anomaly detection threshold for each acoustic feature in the set; obtain the feature value of each acoustic feature; if the obtained feature value satisfies the threshold conditions corresponding to a preset number of acoustic features, determine that the device under test has an operating anomaly; or, Invoke a machine learning model that matches the device type and perform the step of identifying the operating status of the device under test based on the acoustic signal.

11. The method according to any one of claims 1 to 10, characterized in that, The method further includes: A structured detection record is generated; the structured detection record is obtained by encapsulating at least one of the following: first scene information of the target scene, candidate detection area of ​​the device under test, thermal anomaly data reflecting the surface temperature of the device under test, acoustic features of the acoustic signal, parameter information of the PRPD spectrum, or the operating status identification result; Based on the structured detection records and industry knowledge, a prompt text is constructed; the prompt text includes at least one of the following: second scene information of the target scene, output constraint information, or abnormal evaluation information of the tested device; the second scene information has a greater amount of information than the first scene information; Using the prompt text as input to the large language model, the target text is generated by the large language model; wherein, the target text includes at least one of the following: an overview of the current operation status identification, explanatory information of the operation status identification results, explanatory information on the risk level, and processing and retesting suggestions.

12. The method according to claim 11, characterized in that, The step of using the prompt text as input to a large language model and generating target text through the large language model includes: Using the prompt text as input to the large language model, the target text that conforms to the text output framework is generated through the large language model; The text output framework is matched with the target scene.

13. The method according to claim 11, characterized in that, The method further includes: Get user questions; The user's question and the structured detection record are input into the large language model, and the response content from the large language model is obtained. Output the reply content.

14. The method according to claim 11, characterized in that, The display screen shows a report generation button; the method further includes: In response to the triggering operation of the report generation button, a device status report is output; The equipment status report includes at least one of the following: The at least one image; Sound source location coordinates, which are used to indicate the position of the device under test in the target scene; Temperature trend curve, which is used to describe the change trend of the surface temperature of the device under test over time; The structured detection record; The target text.

15. A device for identifying the operating status of equipment, characterized in that, The device includes: The first acquisition module is configured to acquire acoustic signals collected in the target scene; The first generation module is configured to generate at least one image based on the acoustic signal, using the device type of the device under test in the target scene as a constraint. The identification module is configured to identify the operating status of the device under test based on the acoustic signal, and obtain an operating status identification result; wherein, the at least one image and the operating status identification result are used to characterize the current operating status of the device under test from different perspectives; The output module is configured to output the running status recognition result and visualize it based on the at least one image.

16. A device operating status identification component, characterized in that, The component includes a processor and a memory, the memory storing computer program code, which is loaded and executed by the processor to implement the device operating status identification method as described in any one of claims 1 to 14.

17. An imager, characterized in that, The imager includes: an acoustic signal acquisition module, a thermal imaging module, and a main body; The acoustic signal acquisition module is fixedly connected to the main body; the thermal imaging module is snapped into the main body; the main body is electrically connected to both the acoustic signal acquisition module and the thermal imaging module. The main body includes a device operation status identification component, which is configured to: perform the device operation status identification method as described in any one of claims 1 to 14 based on one or more of the acoustic signals acquired by the acoustic signal acquisition module or the thermal imaging signals acquired by the thermal imaging module.

18. A computer-readable storage medium, characterized in that, The storage medium stores computer program code, which is loaded and executed by a processor to implement the device operating status identification method as described in any one of claims 1 to 14.

19. A computer program product, characterized in that, The computer program product includes computer program code stored in a computer-readable storage medium. The processor of the device operation status identification component reads the computer program code from the computer-readable storage medium and executes the computer program code, causing the imager to perform the device operation status identification method as described in any one of claims 1 to 14.

Images