A partial discharge positioning method and system based on infrared and acoustic imaging fusion

Abstract

The present application relates to the technical field of industrial inspection and monitoring, and in particular to a partial discharge positioning method and system based on the fusion of infrared and acoustic imaging, which includes: obtaining a visible light image; dynamically selecting a corresponding frequency band range based on an acoustic selection rule, and obtaining an acoustic imaging layer; adding the acoustic imaging layer to the visible light image to obtain an acoustic monitoring map and generate a first positioning result; obtaining an infrared imaging layer containing the range of the monitored object, copying the visible light image and adding the infrared imaging layer to obtain an infrared monitoring map and generate a second positioning result; displaying the acoustic monitoring map and the infrared monitoring map on the same screen; fusing the first positioning result and the second positioning result to determine whether there are overlapping positioning points, calculating the expert rewards corresponding to the overlapping positioning points, and judging whether it is a fault scenario based on the expert rewards to perform positioning alarms. The present application has the effect of reducing interference to partial discharge monitoring in complex environments and realizing visualization of partial discharge position positioning.

Description

Technical Field

[0001] The present application relates to the technical field of industrial inspection and monitoring, and in particular to a local discharge positioning method and system based on infrared and acoustic imaging fusion. Background Art

[0002] In industrial environments, it is often necessary to inspect the operation of each device to determine whether the equipment has any safety hazards such as failure. If applied to power systems, inspections can be used to identify potential partial discharge fault points.

[0003] Currently, handheld ultrasonic monitors are often used to locate partial discharge fault points during inspections. When partial discharge occurs, certain ultrasonic waves will be generated. In some cases, the sound waves corresponding to the fault can be heard by the human ear. However, in some noisy environments or when the overhead power lines are high above the ground, ultrasonic detectors are required for positioning and detection.

[0004] However, the current ultrasonic monitor has certain problems. Due to its low monitoring accuracy, it cannot effectively and accurately monitor the sound waves corresponding to partial discharge faults in a more complex industrial environment. During the monitoring process, there will be a large amount of environmental interference and reflection interference that will affect the real-time monitoring results. At the same time, after the current ultrasonic monitor detects the ultrasonic wave corresponding to the suspected fault, it can only inform the operator of the suspected fault and the corresponding position of the fault, but it cannot accurately display and inform the user which overhead line or which position on the overhead line it is. The operator still needs to constantly adjust the position and angle according to the ultrasonic monitor to find the location of the fault point, which is a cumbersome operation. Summary of the invention

[0005] In order to reduce interference to partial discharge positioning in complex environments and realize visualization of partial discharge position positioning, the present application provides a partial discharge positioning method and system based on infrared and acoustic imaging fusion.

[0006] In the first aspect, the present application provides a local discharge positioning method based on infrared and acoustic imaging fusion, which adopts the following technical solution:

[0007] A partial discharge positioning method based on infrared and acoustic imaging fusion includes the following steps:

[0008] Acquire a visible light image including the range of the monitored object;

[0009] Dynamically select the corresponding frequency band range based on the acoustic selection rule, and obtain the acoustic imaging layer including the monitoring object range;

[0010] Adding the acoustic imaging layer to the visible light image to obtain an acoustic monitoring map, and generating a first positioning result based on the acoustic monitoring map;

[0011] Acquire an infrared imaging layer including the range of the monitored object, copy the visible light image and add the infrared imaging layer to obtain an infrared monitoring map, and generate a second positioning result based on the infrared monitoring map;

[0012] Displaying the acoustic monitoring image and the infrared monitoring image on the same screen;

[0013] The first positioning result and the second positioning result are integrated to determine whether there are overlapping positioning points, the expert rewards corresponding to the overlapping positioning points are calculated, and based on the expert rewards, it is determined whether it is a fault scenario to issue a positioning alarm.

[0014] In some of the embodiments, dynamically selecting a corresponding frequency band range based on an acoustic selection rule and obtaining an acoustic imaging layer including a monitoring object range includes the following steps:

[0015] Obtain a first frequency band range and sequentially select between preset upper and lower limits of the range to obtain a first pre-processing layer containing acoustic cloud feature points corresponding to a plurality of different ranges;

[0016] Acquire a second frequency band range and sequentially select between preset upper and lower limits of the range to acquire a second pre-processing layer containing acoustic cloud feature points corresponding to a plurality of different ranges;

[0017] The first frequency band range is larger than the second frequency band range.

[0018] In some of the embodiments, obtaining an acoustic imaging layer including a range of a monitoring object includes the following steps:

[0019] Determine whether the frequency band scenario is known;

[0020] If so, obtaining the first pre-processed layer and the second pre-processed layer matching the known frequency band scene;

[0021] Comparing the first preprocessing layer and the second preprocessing layer to obtain the overlapping acoustic cloud feature points;

[0022] Acquire a layer set as the known frequency band scene containing a monitoring object range as a comparison graph, and verify the first feature data of the acoustic cloud feature point with the acoustic cloud feature point in the comparison graph, wherein the first feature data includes position data and color data;

[0023] After the verification is passed, the acoustic imaging layer is generated based on the acoustic cloud feature points;

[0024] If not, traverse all the first pre-processing layers, and compare each of the first pre-processing layers with each of the second pre-processing layers within the first frequency band to determine whether there are overlapping sound cloud feature points;

[0025] A number of overlapping acoustic cloud feature points are marked, and the first feature data corresponding to the marked acoustic cloud feature points are compared in the visible light image. When the comparison is successful, the corresponding second preprocessing layer containing only the acoustic cloud feature points is used as the acoustic imaging layer.

[0026] In some of the embodiments, obtaining an acoustic imaging layer including a range of a monitoring object further includes the following steps:

[0027] Acquire the selected directional display range, take the display position in the acoustic imaging layer that meets the directional display range as the effective sound pickup range, and take the display position in the acoustic imaging layer that does not meet the directional display range as the invalid sound pickup range;

[0028] In the acoustic imaging layer, the acoustic cloud feature points within the effective sound pickup range are displayed, and the acoustic cloud feature points in the invalid sound pickup range are eliminated.

[0029] In some of the embodiments, the acoustic imaging layer is added to the visible light image to obtain an acoustic monitoring map, and the first positioning result is generated based on the acoustic monitoring map, further comprising the following steps:

[0030] Overlaying the acoustic imaging layer on the visible light image to obtain the acoustic monitoring map;

[0031] The first feature data of the acoustic cloud feature point in the acoustic monitoring image is analyzed to generate a first positioning result, wherein the first positioning result includes a positional relationship between the acoustic cloud feature point and the monitored object and a potential fault type corresponding to the acoustic cloud feature point.

[0032] In some of the embodiments, the acoustic imaging layer is added to the visible light image to obtain an acoustic monitoring map, and the first positioning result is generated based on the acoustic monitoring map, further comprising the following steps:

[0033] Acquiring a plurality of the acoustic monitoring images corresponding to different monitoring positions, and determining whether the first positioning results corresponding to the acoustic cloud feature points in the plurality of the acoustic monitoring images match;

[0034] If there is no match, the first positioning result is invalid and the acoustic monitoring map is generated again;

[0035] If they match, the first positioning result is valid.

[0036] In some embodiments, obtaining an infrared imaging layer including the range of the monitored object, copying the visible light image and adding the infrared imaging layer to obtain an infrared monitoring map, and generating a second positioning result based on the infrared monitoring map include the following steps:

[0037] The infrared imaging layer formed by the thermal imaging feature points obtained by the infrared monitoring device is used to obtain second feature data of each of the thermal imaging feature points, wherein the second feature data includes position data and color data;

[0038] Overlaying the infrared imaging layer on the visible light image to obtain the infrared monitoring image;

[0039] The second feature data of the thermal imaging feature point in the infrared monitoring image is analyzed to generate a second positioning result, wherein the second positioning result includes a positional relationship between the thermal imaging feature point and the monitored object and a potential fault type corresponding to the thermal imaging feature point.

[0040] In some embodiments, fusing the first positioning result and the second positioning result to determine whether there is an overlapping positioning point includes the following steps:

[0041] Generating a virtual image including an object contour based on the visible light image, wherein the virtual image includes a plurality of virtual monitoring bodies formed by contour lines;

[0042] Based on the first positioning result and the second positioning result, an acoustic simulation point and an infrared simulation point are generated in the virtual monitoring body, and coordinate information of the acoustic virtual point and the infrared simulation point are obtained and compared to determine whether the positions overlap;

[0043] If there is position overlap, it is defined as the occurrence of overlapping positioning points.

[0044] In some embodiments, calculating the expert reward corresponding to the coincident positioning point and judging whether it is a fault scenario based on the expert reward to perform positioning alarm includes the following steps:

[0045] Analyzing a monitoring type based on the contour of the monitoring object in the visible light image;

[0046] Acquire a corresponding fault reasoning group based on the monitoring type in a preset expert database, wherein the fault reasoning group includes frequency band ranges and temperature ranges corresponding to different faults;

[0047] Acoustic comparison: traverse the frequency band ranges in the fault reasoning group and compare them with the color data corresponding to the coincident positioning points, and select the fault with the highest comparison similarity as the reward party to obtain the corresponding first reward;

[0048] Infrared comparison: traverse each temperature range in the fault reasoning group and compare with the color data corresponding to the coincident positioning point, and select the fault with the highest comparison similarity as the reward party to obtain the corresponding second reward;

[0049] Determine whether the faults respectively corresponding to the acoustic comparison and the infrared comparison are the same as the potential fault types corresponding to the first positioning result and the second positioning result;

[0050] A confidence coefficient is obtained based on the same number, and the sum of the first reward and the second reward is multiplied by the confidence coefficient to calculate the expert reward.

[0051] In the second aspect, the present application provides a partial discharge positioning system based on infrared and acoustic imaging fusion, which adopts the following technical solution:

[0052] A partial discharge positioning system based on the fusion of infrared and acoustic imaging is used to implement the above method.

[0053] The technical solution provided by the embodiments of the present application has the following technical effects:

[0054] The acoustic cloud image obtained by acoustic forming is combined with the visible light image, and the thermal imaging obtained by infrared temperature measurement is combined with the visible light image. Through the acoustic-infrared fusion method, the infrared temperature measurement results can assist in verifying the acoustic positioning results in some complex and multi-interference environments. At the same time, the acoustic positioning and infrared positioning are displayed in the visible light image. The operator can know the specific location distribution of the possible partial discharge fault in the actual scene. The user no longer needs to constantly change the monitoring position and direction to find the specific fault location, but can quickly locate the fault to the specific overhead line position and pipeline position. Finally, the fault type can be intelligently analyzed in the acoustic monitoring results and infrared monitoring results through the scoring of the expert database, which is convenient for the operator to carry out maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a schematic diagram of the steps of a partial discharge positioning method based on the fusion of infrared and acoustic imaging provided in this embodiment. DETAILED DESCRIPTION

[0056] To more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. However, it should be understood by those of ordinary skill in the art that the present application can be implemented without these details. In some cases, in order to avoid unnecessary descriptions that make various aspects of the present application obscure, well-known methods, processes, systems, components and / or circuits that have been described at a higher level will not be described in detail. For those of ordinary skill in the art, it is obvious that various changes can be made to the embodiments disclosed in the present application, and without departing from the principles and scope of the present application, the general principles defined in the present application can be applied to other embodiments and application scenarios. Therefore, the present application is not limited to the embodiments shown, but conforms to the broadest scope consistent with the scope claimed for protection of the present application.

[0057] In the description of this application, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used to distinguish the technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0058] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples.

[0059] like Figure 1 As shown, the embodiment of the present application discloses a local discharge positioning method based on infrared and acoustic imaging fusion, comprising the following steps:

[0060] S100, obtaining a visible light image including a range of a monitoring object.

[0061] The visible light image is acquired by taking pictures with an optical camera on a handheld monitoring instrument. The monitoring object range is characterized by the position range corresponding to a certain area radiated outward from the location of the monitoring object.

[0062] Monitoring objects may include power grid overhead lines, pipelines, industrial equipment, etc.

[0063] S200, dynamically selecting a corresponding frequency band range based on an acoustic selection rule, and obtaining an acoustic imaging layer including a monitoring object range.

[0064] The handheld monitoring instrument is provided with a sound receiving matrix composed of several microphones, which realizes the acquisition of acoustic features by setting the received frequency band range and filtering the sound frequency interference outside the frequency band range. In the embodiment of the present application, the number of microphones is 128.

[0065] The acoustic selection rule is to manually or automatically dynamically select the sound frequency range that needs to be received in different industrial scenarios through a preset selection method. In the embodiment of the present application, a bandwidth of 2kHz-55kHz is supported, and the sound pressure level range is 30db-120db.

[0066] The microphone matrix on the handheld monitoring instrument is combined with beamforming technology to obtain acoustic images in different frequency bands, and the sound source point is displayed in color through pseudo-color technology to obtain the acoustic cloud image.

[0067] Beamforming is an audio signal processing technology that adjusts the microphone's reception pattern so that it can better capture sound from a specific direction. This technology can reduce the impact of ambient noise and improve the clarity of sound. Beamforming technology relies on the collaborative work of multiple microphone elements. Through calculation and phase control, the microphone can adjust its reception pattern so that the sound beam is more focused in a specific direction and less sound is captured in other directions.

[0068] S300, adding an acoustic imaging layer to a visible light image to obtain an acoustic monitoring map, and generating a first positioning result based on the acoustic monitoring map.

[0069] In order to enable the operator to clearly know the position of the sound source point located in the acoustic imaging layer, the present application combines the acoustic imaging layer with the video footage captured in real time by the camera to fuse the sound source distribution data with the visible light image to obtain an acoustic monitoring map. The acoustic monitoring map is displayed on the display screen of the monitoring instrument, and the user can know exactly where the sound is coming from.

[0070] The first positioning result is characterized by the result of positioning the partial discharge fault through acoustic display data.

[0071] S400, obtaining an infrared imaging layer including a range of a monitoring object, copying a visible light image and adding an infrared imaging layer to obtain an infrared monitoring map, and generating a second positioning result based on the infrared monitoring map.

[0072] The infrared imaging picture corresponding to the monitoring object range is obtained through the external infrared monitoring module. The infrared monitoring equipment is used to monitor the heat temperature emitted by objects within a certain range.

[0073] The color depth of the infrared imaging layer represents the heat emitted by the corresponding object. The infrared imaging layer is fused with the visible light image so that the user can clearly see where the heating abnormality exists based on the image.

[0074] The second positioning result is characterized by the positioning result of the partial discharge fault through infrared thermal imaging display data.

[0075] S500, displays acoustic monitoring images and infrared monitoring images on the same screen.

[0076] The acoustic monitoring image and the infrared monitoring image are displayed simultaneously through the display screen of the monitoring instrument. The display method can be split along the width direction of the display screen, and can also be split along the height direction of the display screen.

[0077] S600, integrating the first positioning result and the second positioning result to determine whether there is an overlapping positioning point, calculating the expert reward corresponding to the overlapping positioning point, and determining whether it is a fault scenario based on the expert reward to issue a positioning alarm.

[0078] The first positioning result and the second positioning result are integrated, so that the fault point of partial discharge can be located by combining and comparing the acoustic imaging result and the infrared imaging result. When the location and content of the abnormal situation in the two results are the same or similar, the expert database can be used to match the definition content of different faults to specifically analyze whether a specific fault situation has occurred, and an alarm can be issued in time when a fault situation occurs. In this way, users can quickly foresee fault situations that cannot be discovered in time by human eyes or ears during the inspection process and quickly locate the location of partial discharge faults.

[0079] It should also be noted that the technical solution of the present application is not only applicable to fault location in partial discharge situations, but also to industrial environments such as monitoring of pipeline high-pressure gas leakage and abnormal mechanical vibration, because these scenarios all correspond to the emission of abnormal sound waves and abnormal heat.

[0080] Through the above method, the acoustic cloud image obtained by acoustic forming is combined with the visible light image, and the thermal imaging obtained by infrared temperature measurement is combined with the visible light image. Through the acoustic-infrared fusion method, the infrared temperature measurement results can assist in verifying the acoustic positioning results in some complex and multi-interference environments. At the same time, the acoustic positioning and infrared positioning are displayed in the visible light image. The operator can specifically know the specific location distribution of the possible partial discharge fault in the actual scene. The user no longer needs to constantly change the monitoring position and direction to find the specific fault location, but quickly locate the fault to the specific pipeline location or overhead line location. Finally, the fault type can be intelligently analyzed in the acoustic monitoring results and infrared monitoring results through the scoring of the expert database, which is convenient for the operator to carry out maintenance.

[0081] In some other embodiments, dynamically selecting a corresponding frequency band range based on an acoustic selection rule and obtaining an acoustic imaging layer including a monitoring object range includes the following steps:

[0082] S210, obtaining a first frequency band range and sequentially selecting within preset upper and lower limits of the range to obtain a first pre-processing layer containing acoustic cloud feature points corresponding to a plurality of different ranges.

[0083] First, a larger frequency band is selected, and the sound frequency is selected in turn in the preset upper and lower limits to obtain the size. Specifically, the first frequency band can be set to 20kHz, so that the upper and lower limits of 0-55kHz are selected respectively. The selection can be based on the endpoint-to-endpoint method, such as 0-20kHz, 20-40kHz, 40-55kHz, or a certain overlapping area can be allowed when selecting, such as 0-20kHz, 10-30kHz, 20-40kHz, 30-50kHz, etc.

[0084] At the same time, each time a range value is selected, one or more corresponding acoustic cloud images within this frequency band are quickly obtained, and each acoustic cloud image contains acoustic cloud feature points corresponding to different sound source points or reflected virtual images appearing after sound reflection.

[0085] Each acoustic cloud image corresponds to the first pre-processing layer.

[0086] The layer selected by the larger frequency band is represented by the corresponding result obtained after coarse-precision audio filtering. When the sound frequencies of multiple sound sources in the picture differ greatly, if a smaller frequency band is directly selected for monitoring, the sound frequency corresponding to the fault may not be effectively monitored. Therefore, the first step of sound source screening is performed over a large range.

[0087] S220, obtaining a second frequency band range and sequentially selecting within preset upper and lower limits of the range to obtain a plurality of second pre-processing layers corresponding to different ranges and containing acoustic cloud feature points.

[0088] Since the acoustic cloud map selected by the first frequency band range will contain more noise interference, in order to further reduce and eliminate the interference of environmental noise on the monitoring effect, it is necessary to obtain multiple acoustic cloud maps in the upper and lower limits through a smaller frequency band range. The acoustic cloud map here still contains acoustic cloud feature points corresponding to the reflected virtual images of different sound source points or sound reflections. In the embodiment of the present application, the second frequency band range can be 10kHz.

[0089] The first frequency band is larger than the second frequency band.

[0090] In other embodiments, the selected range of the sound pressure level can also be dynamically adjusted. For example, when the sound cloud map obtained in different frequency bands does not match the sound state of the actual scene, or when the on-site environment is relatively noisy, the acquisition range of the sound pressure level can be adjusted to a relatively large value. This is because when the sound pressure levels of multiple sound sources differ greatly, a relatively small sound pressure level dynamic range parameter may cause a large sound source to drown out a small sound source. In an actual environment, whether it is a leak in a pipeline in an industrial plant or a local discharge in an overhead line, the sound is relatively small. At the same time, the location of the sound source is far away from the location of the inspection personnel, so the corresponding sound pressure level is relatively small.

[0091] In some other embodiments, an acoustic imaging layer is added to a visible light image to obtain an acoustic monitoring map, and a first positioning result is generated based on the acoustic monitoring map, including the following steps:

[0092] S230, determining whether the frequency band scenario is known.

[0093] The known frequency band scenario is characterized by the fact that in some situations, some inspection personnel will purposefully monitor partial discharge faults at some locations. At this time, the inspection personnel may know the specific frequency band range corresponding to the fault content they want to monitor, or in the inspection of different industries, because the corresponding sound media and fault content in scenarios such as "gas leakage", "electrical fault", and "mechanical vibration" are quite different, there are also large differences between the sound frequencies in different scenarios. The inspection personnel can roughly know the frequency band range corresponding to the fault.

[0094] If the inspection personnel manually select a scene mode or a specific frequency band range, it is considered that there is a known frequency band scene.

[0095] S231: If so, obtain a first preprocessing layer and a second preprocessing layer that match the known frequency band scene.

[0096] If it exists, the logic of comparing the first preprocessing layer and the second preprocessing layer is to select two layer results that match the known frequency band range and compare the comparison results between the two layer results with the acoustic results monitored in the known frequency band scene again, so as to achieve double verification.

[0097] For example, if the corresponding frequency band range in the known frequency band scenario is 18 kHz-36 kHz, then a first preprocessing layer and a second preprocessing layer corresponding to a frequency band range that includes at least half of the above range are obtained.

[0098] For example, a first preprocessing layer of 20-40kHz and a second preprocessing layer of 20-30kHz are obtained.

[0099] S232: Compare the first preprocessed layer and the second preprocessed layer to obtain overlapping sound cloud feature points.

[0100] The acoustic cloud feature points on the first preprocessing layer are compared with the acoustic cloud feature points on the second preprocessing layer. Due to the different selected frequency band ranges, the sizes and numbers of the acoustic cloud points displayed on different layers are different. In the embodiment of the present application, the selected overlapping acoustic cloud feature points are characterized by an overlap of more than 80% area between the two points.

[0101] The overlapping sound cloud feature points are characterized as objects that match each other in the sound results that have been retained to a large extent through coarse screening and the sound results after most of the environmental noise has been removed through fine screening.

[0102] In some embodiments, if the corresponding frequency band range in the known frequency band scenario is 18kHz-36kHz, and the frequency band range corresponding to the selected second pre-processing layer is 20-30kHz, then if the sound frequency corresponding to the partial discharge fault happens to be between 31-36kHz, then the sound cloud feature point may not exist in the second pre-processing layer. In this case, the above comparison results may be misscreened because the partial discharge fault cannot be corresponded to an overlapping sound cloud feature point.

[0103] Therefore, in order to reduce the occurrence of the above situation, after the pre-processing layer is selected through the known frequency band scene, if the frequency band range corresponding to the known frequency band scene has a frequency band exceeding the threshold size and is not covered by the frequency band range of the pre-processing layer, multiple first pre-processing layers or multiple second pre-processing layers can be selected until most of the frequency band range of the known frequency band scene is selected.

[0104] S233, obtaining a layer containing a monitoring object range in a known frequency band scene as a comparison graph, and verifying the first feature data of the acoustic cloud feature point with the acoustic cloud feature point in the comparison graph.

[0105] At the same time, the frequency band range of the monitoring instrument is adjusted to the frequency band range under the known frequency band scenario and the corresponding acoustic layer is obtained as a comparison chart. When the user tends to choose a certain monitoring mode or monitoring frequency band, it is considered that the range has subjective reference value. At this time, the layer can be further compared with the above-mentioned overlapping sound cloud feature points in terms of feature data.

[0106] The first feature data includes position data and color data. The position data is represented by the position of the acoustic cloud feature point displayed on the screen in different acoustically formed layers, that is, the horizontal and vertical coordinates of the acoustic cloud feature point on the display screen with the central coordinates of the acoustic cloud feature point as a reference. The color data is represented by the sound intensity obtained from the monitoring results of different acoustic cloud feature points. The redder the color, the greater the sound intensity.

[0107] In this way, during the comparison process, when the feature data of the overlapping sound cloud feature points are the same as those of the sound cloud feature points in the known frequency band scene, it can be considered that matching feature points have been found in all three layers. When the three layers correspond to different frequency band ranges, the one or more matching feature points can be considered to be the accurate partial discharge fault location.

[0108] S234, after verification, an acoustic imaging layer is generated based on the acoustic cloud feature points.

[0109] After the above verification process is passed, the acoustic imaging layer is generated according to the matched acoustic cloud feature points.

[0110] If the verification fails, the layer corresponding to the known frequency band scene is preferably selected as the acoustic imaging layer.

[0111] S235 , if not, traverse all first pre-processing layers, and compare each first pre-processing layer with each second pre-processing layer within its first frequency band one by one to determine whether there is an overlapping sound cloud feature point.

[0112] If there is no known frequency band scenario, corresponding comparison and analysis is directly performed through a plurality of first pre-processing layers and second pre-processing layers.

[0113] Specifically, firstly, all the first preprocessing layers are traversed, and the second preprocessing layers in the first frequency band range are screened out from each first preprocessing layer. For example, when the first frequency band range is 20-40kHz, the corresponding second preprocessing layers selected are 20-30kHz and 30-40kHz.

[0114] After all the first pre-processed layers and the corresponding second pre-processed layers are selected, the two matching layers are compared respectively, and all the overlapping feature points in the comparison process are selected and selected. The selected sound cloud feature points represent the sound source content captured in different frequency bands.

[0115] S236, marking a number of overlapping acoustic cloud feature points, comparing the first feature data corresponding to the marked acoustic cloud feature points in the visible light image, and when the comparison is successful, using the corresponding second pre-processed layer containing only the acoustic cloud feature points as the acoustic imaging layer.

[0116] First, the overlapping acoustic cloud feature points in multiple groups of comparisons are marked, and at the same time, the first feature data of the marked multiple acoustic cloud feature points are respectively combined with the visible light image for comparison.

[0117] For example, it can determine whether the location of the acoustic cloud feature point is on the monitored object in the visible light image, and use color data to determine whether the acoustic cloud feature point is jumping, whether the sound intensity is consistent with the sound propagation distance that matches the distance between the operator and the monitored object, etc.

[0118] If the location where the acoustic cloud feature point appears is on the monitored object, the acoustic cloud feature point remains fixed, and the sound intensity corresponding to the color conforms to the distance propagation logic, then the acoustic cloud feature point is considered to be successfully matched. At this time, the second pre-processed layer with less interference containing the acoustic cloud feature point is used as the acoustic imaging layer.

[0119] In some other embodiments, obtaining an acoustic imaging layer including a range of a monitoring object further includes the following steps:

[0120] S240, obtaining the selected directional display range, taking the display position in the acoustic imaging layer that meets the directional display range as the effective sound pickup range, and taking the display position in the acoustic imaging layer that does not meet the directional display range as the invalid sound pickup range.

[0121] In some scenarios, when conducting partial discharge fault inspections, operators may want to obtain acoustic feature results in a targeted manner in a small area and shield interference from other sound source points outside the small area.

[0122] Then the user can actively select a directional display range on the display screen. If the directional display range is smaller than the display range of the screen, then the area on the display screen within the directional display range is the effective sound pickup range, and the area on the display screen that is not within the directional display range is the invalid sound pickup range.

[0123] S241, in the acoustic imaging layer, displaying the acoustic cloud feature points within the effective sound pickup range and eliminating the acoustic cloud feature points within the invalid sound pickup range.

[0124] In the final acoustic imaging layer, only the acoustic cloud feature points in the effective pickup range are displayed, while the acoustic cloud feature points in the invalid pickup range are not displayed.

[0125] In this way, through directional pickup, a directional display box appears in the sound and image display area, and the sound source is located in the box area to find the sound source point. The application scenario of this function is that there are multiple sound source points around the target to be measured. The interference of sounds in other locations can be shielded, and acoustic monitoring can only be performed on the places of interest.

[0126] In some other embodiments, an acoustic imaging layer is added to a visible light image to obtain an acoustic monitoring map, and a first positioning result is generated based on the acoustic monitoring map, including the following steps:

[0127] S310, overlaying the acoustic imaging layer on the visible light image to obtain an acoustic monitoring image.

[0128] First, the acoustic imaging layer and visible light image are preprocessed, including time synchronization, denoising, interpolation and normalization of the acoustic imaging layer.

[0129] The edge features of the visible light image and acoustic imaging layer are extracted and matched using the SIFT algorithm or ORB algorithm, and then the features are aligned through affine transformation to achieve coverage.

[0130] The acoustic monitoring image can also be superimposed on the visible light image as a transparent layer (50%-80% transparency), the formula is: , where a represents the transparency value, X1 is the acoustic cloud pixel, X2 is the visible light pixel, and X3 is the fusion pixel.

[0131] S320, analyzing first feature data of an acoustic cloud feature point in the acoustic monitoring image to generate a first positioning result, wherein the first positioning result includes a positional relationship between the acoustic cloud feature point and the monitored object and a potential fault type corresponding to the acoustic cloud feature point.

[0132] The first feature data is analyzed, and the positional coordinates of the acoustic cloud feature points on the screen are compared with the positional coordinates of the monitored object on the image to generate a positional relationship between the acoustic cloud feature points and the monitored object.

[0133] At the same time, the type of the monitored object is analyzed according to the contour of the monitored object, and the potential fault type that may correspond to the monitored object in the frequency band and sound intensity is comprehensively analyzed in combination with the sound intensity information corresponding to the color data of the sound cloud feature point and the frequency band range corresponding to the sound cloud feature point.

[0134] In some other embodiments, an acoustic imaging layer is added to a visible light image to obtain an acoustic monitoring map, and a first positioning result is generated based on the acoustic monitoring map, further comprising the following steps:

[0135] S330, obtaining a plurality of acoustic monitoring images corresponding to different monitoring positions, and determining whether the first positioning results corresponding to the acoustic cloud feature points in the plurality of acoustic monitoring images match.

[0136] S340: If there is no match, the first positioning result is invalid and the acoustic monitoring map is generated again.

[0137] S350: If there is a match, the first positioning result is valid.

[0138] In some cases, the acoustic cloud feature points monitored in the acoustic cloud map may be the actual sound source points, or they may be the reflected virtual images detected after the sound waves emitted by the sound source are reflected in some media. In order to eliminate the interference of reflection, the operator can change his position and angle to capture the acoustic cloud map at different angles while keeping the monitored object range displayed on the screen.

[0139] The acoustic cloud feature points in the acoustic monitoring images at different positions are compared to determine whether the first positioning result has changed. If the first positioning results in multiple images match, that is, the same sound source can be captured at multiple angles, then the acoustic cloud feature point corresponding to the sound source is generally the actual sound source position. If the acoustic cloud feature points at different positions and angles have changed in the first positioning result, such as disappearance or color fluctuations, it is considered that the virtual image of the reflected sound source has drifted or even disappeared at different positions.

[0140] In some other embodiments, obtaining an infrared imaging layer including a range of a monitoring object, copying a visible light image and adding the infrared imaging layer to obtain an infrared monitoring map, and generating a second positioning result based on the infrared monitoring map include the following steps:

[0141] S410, obtaining an infrared imaging layer composed of thermal imaging feature points based on the infrared monitoring device, and obtaining second feature data of each thermal imaging feature point, the second feature data including position data and color data.

[0142] An infrared imaging layer covering the range of the monitored object is obtained through infrared monitoring equipment, on which different color depths are displayed according to the emitted temperature of different objects at the measured position, wherein the infrared imaging layer contains the contours of objects in the measured area.

[0143] The second characteristic data of each thermal imaging feature point in the infrared imaging layer is extracted. The thermal imaging feature point is characterized by a position point where the temperature is significantly higher than the preset temperature, resulting in an extremely high imaging color depth. Each second characteristic data includes position data and color data.

[0144] The position data is represented by the specific horizontal and vertical coordinates corresponding to the position of the feature point displayed on the screen, and the color data is represented by different color depths and corresponding temperature information.

[0145] S420, overlay the infrared imaging layer on the visible light image to obtain an infrared monitoring image.

[0146] In the same way as the fusion of the acoustic imaging layer, the infrared imaging layer is also overlaid on the visible light image to achieve the combination of infrared and visible light effects to obtain an infrared monitoring map.

[0147] S430, analyzing the second feature data of the thermal imaging feature point in the infrared monitoring image to generate a second positioning result, wherein the second positioning result includes the positional relationship between the thermal imaging feature point and the monitored object and the potential fault type corresponding to the thermal imaging feature point.

[0148] The second characteristic data of each thermal imaging feature point in the infrared monitoring image is analyzed to obtain the second positioning result of each feature point under infrared monitoring. Specifically, the positional relationship between the thermal imaging feature point and the monitored object is obtained based on the calculation between the position coordinates of the thermal imaging feature point and the edge coordinates of the monitored object in the infrared monitoring image. Through this positional relationship, it can be determined whether the heat source is on the monitored object. It also includes whether the heating temperature analyzed based on the color data of the thermal imaging feature point matches the heating condition of the fault corresponding to the type of monitored object to analyze the potential corresponding fault type.

[0149] In some other embodiments, fusing the first positioning result and the second positioning result to determine whether there is an overlapping positioning point includes the following steps:

[0150] S610, generating a virtual image including an object contour based on the visible light image, wherein the virtual image includes a plurality of virtual monitoring objects formed by contour lines.

[0151] When fusing and analyzing the acoustic cloud feature points corresponding to the first positioning result and the thermal imaging feature points corresponding to the second positioning result, it is first necessary to determine whether there is any overlap in position between the two. This is because, in general, for most industrial scenarios, such as local discharge, local gas leakage, and local mechanical surge, which are accompanied by abnormal sound and abnormal heat, the sound source position and the heat source position are generally in the same position.

[0152] In this case, firstly, by determining the position overlap, further analysis and judgment can be made as to whether a fault occurs and the specific position of the fault.

[0153] Specifically, firstly, the contour of each object in the visible light image is selected according to the image processing technology and a virtual structure is formed based on the contour line. Each pixel point on the contour line corresponds to specific coordinate information.

[0154] S620: Generate acoustic simulation points and infrared simulation points in the virtual monitoring body based on the first positioning result and the second positioning result, obtain coordinate information of the acoustic virtual points and the infrared simulation points, and compare them to determine whether the positions overlap.

[0155] The center point coordinate information is selected according to the position data of the acoustic cloud feature point in the first positioning result, and the center point coordinate information is selected according to the position data of the thermal imaging feature point in the second positioning result.

[0156] The positions corresponding to the coordinate information of the two center points correspond to the acoustic simulation point and the infrared simulation point respectively. The position coordinates of the two points are compared. When the difference between the horizontal and vertical coordinates of the two points is less than the preset value, it is considered that the distance between the two positions is small. At this time, it is considered that the acoustic cloud feature point and the thermal imaging feature point coincide with each other. The preset value is generally the sum of the radius of the acoustic cloud feature point and the thermal imaging feature point.

[0157] S630: If there is position overlap, it is defined as the occurrence of overlapped positioning points.

[0158] When the two are overlapping positioning points, the sound source point and the heat point are considered to be at the same position. Based on these two overlapping positioning points, combined with the visual fusion on the visible light image, the operator can easily locate the fault position.

[0159] In some other embodiments, calculating the expert reward corresponding to the coincident positioning point and judging whether it is a fault scenario based on the expert reward to perform positioning alarm includes the following steps:

[0160] S640: Analyze the monitoring type based on the contour of the monitoring object in the visible light image.

[0161] First, the type of the monitored object is determined based on the edge contour line of the monitored object combined with edge detection technology. Because the appearance of the monitored objects in different detection scenarios is often quite different, such as overhead power lines, pipelines, substation equipment, etc., the type of the monitored object can be quickly analyzed through the contour line.

[0162] S641, obtaining a corresponding fault inference group based on the monitoring type in a preset expert database, where the fault inference group includes frequency band ranges and temperature ranges corresponding to different faults.

[0163] An expert database is preset, which stores all possible fault conditions of different monitoring object types. Multiple faults of each monitoring type are integrated into a fault reasoning group, and each fault corresponds to the sound frequency band range and heating temperature range under the fault condition. For example, the fault types of partial discharge of overhead power lines include suspended discharge, surface discharge, corona discharge, etc. Different fault types have different sound frequency bands and different heating temperatures. The expert database is used to store and integrate the above fault conditions.

[0164] S642, acoustic comparison: traverse each frequency band range in the fault reasoning group and compare it with the color data corresponding to the overlapping positioning point, and select the fault with the highest comparison similarity as the reward party to obtain the corresponding first reward.

[0165] First, all faults in the fault inference group that matches the monitoring type are traversed, and the frequency band range corresponding to each fault is obtained and compared with the frequency band size analyzed by the color data of the coincident positioning point. Finally, after all faults are compared, the fault with the closest comparison result is selected to give a reward. The reward is characterized by the corresponding fault being the most likely result analyzed from the feature data of the coincident positioning point.

[0166] S643, infrared comparison: traverse each temperature range in the fault reasoning group and compare it with the color data corresponding to the coincident positioning point, and select the fault with the highest comparison similarity as the reward party to obtain the corresponding second reward.

[0167] Secondly, traverse all faults in the fault reasoning group that matches the monitoring type, and obtain the temperature range corresponding to each fault and compare it with the heating temperature analyzed by the color data of the coincident positioning point. Finally, after all faults are compared, the fault with the closest comparison result is selected and rewarded. The reward is represented by the corresponding fault being the most likely result analyzed from the feature data of the coincident positioning point.

[0168] In actual industrial inspection scenarios, when a fault occurs, a corresponding abnormal sound will definitely be generated, but whether heat will be generated depends on the specific type of the monitored object and the fault condition. Therefore, in the actual expert library comparison and reward process, the first reward is higher than the second reward, which means that the analysis weight of the acoustic results is greater than the analysis weight of the heating results.

[0169] S644, determining whether the faults corresponding to the acoustic comparison and the infrared comparison are both the same as the potential fault types corresponding to the first positioning result and the second positioning result.

[0170] When both the acoustic comparison and the infrared comparison are completed, it is determined whether the faults that respectively obtain the first reward and the second reward in the two comparison processes are the same as the potential fault types corresponding to the first positioning result and the second positioning result.

[0171] The potential fault types of the first positioning result and the second positioning result are predicted faults obtained after only independently referring to the acoustic data and the thermal imaging data, and each positioning result may correspond to multiple predicted potential fault types.

[0172] The acoustic comparison and infrared comparison are both based on the frequency band range and temperature range corresponding to a fault in the expert database. This allows the results of independent analysis and overall analysis to be compared for further verification.

[0173] S645, obtaining a confidence coefficient based on the same number, and multiplying the sum of the first reward and the second reward by the confidence coefficient to calculate the expert reward.

[0174] After the results of the acoustic comparison and the infrared comparison are compared with the potential fault types respectively, the number of successful comparisons of the infrared and acoustic results is determined, and a confidence coefficient is generated based on the number. When the acoustic result and the infrared result are identically matched, it is characterized that both the fault result analyzed based on the acoustics and the fault result analyzed based on the infrared are correct based on the judgment of the expert database, then the confidence coefficient is large at this time, and when only one of the results is the same, the corresponding confidence coefficient is small. In the embodiment of the present application, when the number of identical results is 2, the size of the confidence coefficient is 1.8, when the number of identical results is 1, the size of the confidence coefficient is 1.2, and when the number of identical results is 0, it is characterized that both the acoustic result and the infrared result are considered to have no corresponding fault after comparison by the expert database, then the confidence coefficient is 0.5.

[0175] After obtaining the confidence coefficient, add the first reward and the second reward obtained in the two comparison processes, and then multiply the result of the addition by the confidence coefficient to calculate the expert reward.

[0176] The size of the expert reward can be used to determine whether there is a fault and the specific type of fault.

[0177] When a fault is detected, the monitoring instrument issues a corresponding alarm to remind the operator that a fault exists. At the same time, it quickly informs the inspection personnel of the location of the fault by integrating the display content of the visible light image and the dual comparison display of acoustics and infrared.

[0178] The embodiments of the present application also disclose a local discharge positioning system based on the fusion of infrared and acoustic imaging, which is used to implement the above method.

[0179] The implementation principle is:

[0180] The acoustic cloud image obtained by acoustic forming is combined with the visible light image, and the thermal imaging obtained by infrared temperature measurement is combined with the visible light image. Through the acoustic-infrared fusion method, the infrared temperature measurement results can assist in verifying the acoustic positioning results in some complex and multi-interference environments. At the same time, the acoustic positioning and infrared positioning are displayed in the visible light image. The operator can know the specific location distribution of the possible partial discharge fault in the actual scene. The user no longer needs to constantly change the monitoring position and direction to find the specific fault location, but can quickly locate the fault to the specific overhead line location or pipeline location. Finally, the fault type can be intelligently analyzed in the acoustic monitoring results and infrared monitoring results through the scoring of the expert database, which is convenient for the operator to carry out maintenance.

[0181] It should be understood that although the steps in the flowchart of the accompanying drawings are shown in sequence according to the instructions of the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise clearly stated in this document, the execution of these steps is not strictly limited in order and can be performed in other orders.

[0182] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A local discharge positioning method based on infrared and acoustic imaging fusion, characterized in that: The following steps are involved: Acquire a visible light image including the range of the monitored object; Dynamically select the corresponding frequency band range based on the acoustic selection rule, and obtain the acoustic imaging layer containing the monitoring object range, including: Obtain a first frequency band range and sequentially select between preset upper and lower limits of the range to obtain a first pre-processing layer containing acoustic cloud feature points corresponding to a plurality of different ranges; Acquire a second frequency band range and sequentially select between preset upper and lower limits of the range to acquire a second pre-processing layer containing acoustic cloud feature points corresponding to a plurality of different ranges; Wherein, the first frequency band range is greater than the second frequency band range; Adding the acoustic imaging layer to the visible light image to obtain an acoustic monitoring map, and generating a first positioning result based on the acoustic monitoring map; Acquire an infrared imaging layer including the range of the monitored object, copy the visible light image and add the infrared imaging layer to obtain an infrared monitoring map, and generate a second positioning result based on the infrared monitoring map; Displaying the acoustic monitoring image and the infrared monitoring image on the same screen; The first positioning result and the second positioning result are integrated to determine whether there is a coincident positioning point, the expert reward corresponding to the coincident positioning point is calculated, and based on the expert reward, it is determined whether it is a fault scenario to perform a positioning alarm. Specifically, Analyzing a monitoring type based on a contour line of the monitoring object in the visible light image; Acquire a corresponding fault reasoning group based on the monitoring type in a preset expert database, wherein the fault reasoning group includes frequency band ranges and temperature ranges corresponding to different faults; Acoustic comparison: traverse the frequency band ranges in the fault reasoning group and compare them with the color data corresponding to the coincident positioning points, and select the fault with the highest comparison similarity as the reward party to obtain the corresponding first reward; Infrared comparison: traverse each temperature range in the fault reasoning group and compare with the color data corresponding to the coincident positioning point, and select the fault with the highest comparison similarity as the reward party to obtain the corresponding second reward; Determine whether the faults respectively corresponding to the acoustic comparison and the infrared comparison are the same as the potential fault types corresponding to the first positioning result and the second positioning result; A confidence coefficient is obtained based on the same number, and the sum of the first reward and the second reward is multiplied by the confidence coefficient to calculate the expert reward.

2. The localization method based on infrared and acoustic imaging fusion according to claim 1 is characterized in that: Obtaining an acoustic imaging layer that includes the scope of the monitored object includes the following steps: Determine whether the frequency band scenario is known; If so, obtaining the first pre-processed layer and the second pre-processed layer matching the known frequency band scene; Comparing the first preprocessing layer and the second preprocessing layer to obtain the overlapping acoustic cloud feature points; Acquire a layer set as the known frequency band scene containing a monitoring object range as a comparison graph, and verify the first feature data of the acoustic cloud feature point with the acoustic cloud feature point in the comparison graph, wherein the first feature data includes position data and color data; After the verification is passed, the acoustic imaging layer is generated based on the acoustic cloud feature points; If not, traverse all the first pre-processing layers, and compare each of the first pre-processing layers with each of the second pre-processing layers within the first frequency band to determine whether there are overlapping sound cloud feature points; A number of overlapping acoustic cloud feature points are marked, and the first feature data corresponding to the marked acoustic cloud feature points are compared in the visible light image. When the comparison is successful, the corresponding second preprocessing layer containing only the acoustic cloud feature points is used as the acoustic imaging layer.

3. The localization method based on infrared and acoustic imaging fusion according to claim 1 is characterized in that: Obtaining an acoustic imaging layer that includes the scope of the monitored object also includes the following steps: Acquire the selected directional display range, take the display position in the acoustic imaging layer that meets the directional display range as the effective sound pickup range, and take the display position in the acoustic imaging layer that does not meet the directional display range as the invalid sound pickup range; In the acoustic imaging layer, the acoustic cloud feature points within the effective sound pickup range are displayed, and the acoustic cloud feature points in the invalid sound pickup range are eliminated.

4. The localization method based on infrared and acoustic imaging fusion according to claim 2 is characterized in that: The acoustic imaging layer is added to the visible light image to obtain an acoustic monitoring map, and a first positioning result is generated based on the acoustic monitoring map, further comprising the following steps: Overlaying the acoustic imaging layer on the visible light image to obtain the acoustic monitoring map; The first feature data of the acoustic cloud feature point in the acoustic monitoring image is analyzed to generate a first positioning result, wherein the first positioning result includes a positional relationship between the acoustic cloud feature point and the monitored object and a potential fault type corresponding to the acoustic cloud feature point.

5. The localization method based on infrared and acoustic imaging fusion according to claim 4 is characterized in that: The acoustic imaging layer is added to the visible light image to obtain an acoustic monitoring map, and a first positioning result is generated based on the acoustic monitoring map, further comprising the following steps: Acquiring a plurality of the acoustic monitoring images corresponding to different monitoring positions, and determining whether the first positioning results corresponding to the acoustic cloud feature points in the plurality of the acoustic monitoring images match; If there is no match, the first positioning result is invalid and the acoustic monitoring map is generated again; If they match, the first positioning result is valid.

6. The localization method based on infrared and acoustic imaging fusion according to claim 2 is characterized in that: Acquiring an infrared imaging layer including the range of the monitored object, copying the visible light image and adding the infrared imaging layer to obtain an infrared monitoring map, and generating a second positioning result based on the infrared monitoring map, including the following steps: The infrared imaging layer formed by the thermal imaging feature points obtained by the infrared monitoring device is used to obtain second feature data of each of the thermal imaging feature points, wherein the second feature data includes position data and color data; Overlaying the infrared imaging layer on the visible light image to obtain the infrared monitoring image; The second feature data of the thermal imaging feature point in the infrared monitoring image is analyzed to generate a second positioning result, wherein the second positioning result includes a positional relationship between the thermal imaging feature point and the monitored object and a potential fault type corresponding to the thermal imaging feature point.

7. The local discharge positioning method based on infrared and acoustic imaging fusion according to claim 4 or 6, characterized in that: Fusion of the first positioning result and the second positioning result to determine whether overlapping positioning points occur includes the following steps: Generating a virtual image including an object contour based on the visible light image, wherein the virtual image includes a plurality of virtual monitoring bodies formed by contour lines; Based on the first positioning result and the second positioning result, an acoustic simulation point and an infrared simulation point are generated in the virtual monitoring body, and coordinate information of the acoustic simulation point and the infrared simulation point are obtained and compared to determine whether the positions overlap; If there is position overlap, it is defined as the occurrence of overlapping positioning points.

8. A partial discharge positioning system based on infrared and acoustic imaging fusion, characterized in that: Used to implement the method described in any one of claims 1 to 7.

Images