Leakage source positioning method and device based on acoustic imaging and visible light image fusion, electronic equipment and storage medium

Abstract

The invention discloses a leakage source positioning method and device based on acoustic imaging and visible light image fusion, electronic equipment and a storage medium, and belongs to the technical field of acoustic imaging, and the method comprises the steps: carrying out the sampling of a sound pressure value of each acoustic signal, and forming a complete sound pressure sampling sequence; sound pressure sampling points located in a preset middle section are selected from the complete sound pressure sampling sequence, and a reference sampling sequence is constructed; calculating a time delay value between the acoustic signals according to the complete sound pressure sampling sequence and the reference sampling sequence; determining the coordinate position of the leakage source in the space by using the time delay value; mapping the space coordinates according to a preset rotation angle and a preset translation vector to obtain pixel coordinates of the leakage source in the image; and carrying out image rendering by taking the pixel coordinate of the leakage source as a center to generate a leakage source image, and fusing the leakage source image with the visible light image to obtain a leakage source fused image. By implementing the method and the device, the problem of low efficiency of positioning the leakage source in the prior art can be solved.

Description

Technical Field

[0001] This invention relates to the field of acoustic imaging technology, and more specifically to a method, apparatus, electronic device, and storage medium for locating leakage sources based on the fusion of acoustic imaging and visible light images. Background Technology

[0002] Gas leaks from pressure equipment or pipelines are a persistent and significant safety hazard during industrial production, storage, and transportation. Leaks first cause media loss and energy waste, leading to direct economic problems; more seriously, leaks of flammable, explosive, or toxic gases can easily trigger fires, explosions, or poisoning accidents, posing a direct threat to production safety and personnel health. Therefore, simply detecting a leak is insufficient; efficiently and accurately locating the leak source is crucial for preventing escalation, guiding emergency response, and implementing rapid repairs. Currently, acoustic imaging is one of the mainstream technologies for leak source location. Acoustic imaging technology uses sensor arrays to collect leak acoustic signals and performs mathematical operations on the signals received by different sensors to calculate the coordinates of the sound source.

[0003] However, existing technologies often require complete calculations on all sound pressure sampling points acquired by sensors when processing data. For example, in cross-correlation calculations, one complete sound pressure sampling sequence is typically compared point-by-point with another complete sound pressure sampling sequence. While this approach theoretically attempts to utilize all data information, it also results in an enormous computational load and consumes massive amounts of computing resources. Furthermore, the edge data located at the beginning and end of the complete sound pressure sampling sequence usually contains edge effects or noise from the acquisition process, contributing very little to the accurate calculation time delay. Including edge data in the calculations in existing technologies not only leads to unnecessary computational resource consumption but also results in overall low efficiency in leak source localization, making it difficult to meet the high requirements for rapid response in industrial sites. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for locating leakage sources based on the fusion of acoustic imaging and visible light images, which can solve the problem of low efficiency in locating leakage sources in the prior art.

[0005] An embodiment of the present invention provides a method for locating leakage sources based on the fusion of acoustic imaging and visible light images, comprising: Multiple acoustic signals of the target area and a visible light image of the target area are acquired; wherein each acoustic signal is acquired by a sensor in the sensor array and corresponds to a sensor in the sensor array; The sound pressure value of each acoustic signal is sampled to generate sound pressure sampling points for each acoustic signal; based on the sound pressure sampling points of each acoustic signal, a complete sound pressure sampling sequence for each acoustic signal is generated. From the complete sound pressure sampling sequence of each acoustic signal, select the sound pressure sampling point located in the preset middle interval as the reference sampling point of each acoustic signal; generate the reference sequence of each acoustic signal based on the reference sampling point of each acoustic signal. Based on the complete sound pressure sampling sequence of each acoustic signal and the reference sequence of each acoustic signal, the time delay value between each acoustic signal is calculated; Based on each time delay value, the spatial coordinates of the leakage source in the target area are calculated; Based on the preset rotation angle and preset translation vector, as well as the spatial coordinates of the leakage source in the target area, coordinate mapping is performed to generate the target pixel coordinates of the leakage source in the target area. Using the target pixel coordinates of the leakage source in the target area as the center, image rendering is performed to generate an image of the leakage source in the target area; The leak source image and the visible light image are fused to generate a fused leak source image.

[0006] Furthermore, the step of calculating the time delay value between each acoustic signal based on the complete sound pressure sampling sequence of each acoustic signal and the reference sequence of each acoustic signal includes: For each acoustic signal, the current acoustic signal is taken as the first acoustic signal, and the other acoustic signals besides the current acoustic signal are taken as the second acoustic signals. For each second acoustic signal, the reference sequence of the first acoustic signal is cross-correlated with the complete sound pressure sampling sequence of the current second acoustic signal to generate a cross-correlation function between the first acoustic signal and the current second acoustic signal; from the cross-correlation function between the first acoustic signal and the current second acoustic signal, the time offset when the cross-correlation function value is the largest is selected as the time delay value between the first acoustic signal and the current second acoustic signal.

[0007] Furthermore, the cross-correlation function between the reference sequence of the first acoustic signal and the complete sound pressure sampling sequence of the current second acoustic signal is calculated using the following formula: In the formula, For the first The acoustic signal and the first The cross-correlation function of the acoustic signals; This is the time offset; For time indexing; For the first The reference sequence of acoustic signals at time index The sound pressure level at that location; For the first The complete sound pressure level sampling sequence of the acoustic signal, after offset, is indexed in time. The sound pressure level at that location.

[0008] Furthermore, the spatial coordinates of the leakage source in the target area are calculated based on each time delay value using the following formula: In the formula, The spatial coordinates of the leakage source in the target area; For the first The spatial coordinates of the sensor corresponding to each acoustic signal; For the first The spatial coordinates of the sensor corresponding to each acoustic signal; For the first The acoustic signal and the first The time delay value of an acoustic signal; The speed of sound.

[0009] Furthermore, the step of generating target pixel coordinates of the leakage source in the target area by performing coordinate mapping based on a preset rotation angle, a preset translation vector, and the spatial coordinates of the leakage source in the target area includes: Two-dimensional coordinate components are extracted from the spatial coordinates of the leakage source in the target area; The two-dimensional coordinate components are rotated according to a preset rotation angle to generate rotated coordinates; The translated coordinates are generated by performing a translation operation on the rotated coordinates according to the preset translation vector. The translated coordinates are used to determine the target pixel coordinates of the leakage source in the target area.

[0010] Furthermore, the step of rendering an image centered on the target pixel coordinates of the leakage source in the target area to generate an image of the leakage source in the target area includes: An image matrix of a preset size is constructed with the target pixel coordinates of the leakage source in the target area as the center; Set the center pixel of the image matrix to a preset maximum intensity value, and set the boundary pixels of the image matrix to a preset minimum intensity value; Based on the maximum intensity value and the minimum intensity value, interpolation calculations are performed on the remaining pixels within the image matrix to generate an intensity distribution map; Based on the intensity distribution map, pseudo-color mapping is performed on the image matrix to generate a leakage source image of the target area.

[0011] Furthermore, the process of fusing the leak source image and the visible light image to generate a fused leak source image includes: Set preset transparency parameters for the leak source image; Based on the transparency parameter, the image of the leakage source is superimposed on the corresponding pixels of the visible light image to generate an intermediate fused image; The intermediate fused image is subjected to contrast enhancement processing to generate a leak source fused image.

[0012] Based on the above method embodiments, the present invention provides corresponding apparatus embodiments.

[0013] One embodiment of the present invention provides a leakage source localization device based on acoustic imaging and visible light image fusion, comprising: a data acquisition module, a sequence generation module, a time delay calculation module, a leakage image generation module, and an image fusion module; The data acquisition module is used to acquire multiple acoustic signals of the target area and a visible light image of the target area; wherein each acoustic signal is acquired by a sensor in the sensor array and corresponds to a sensor in the sensor array; The sequence generation module is used to sample the sound pressure values ​​of each acoustic signal to generate sound pressure sampling points for each acoustic signal; generate a complete sound pressure sampling sequence for each acoustic signal based on the sound pressure sampling points; select sound pressure sampling points located in a preset middle interval from the complete sound pressure sampling sequences of each acoustic signal as reference sampling points for each acoustic signal; and generate a reference sequence for each acoustic signal based on the reference sampling points for each acoustic signal. The time delay calculation module is used to calculate the time delay value between each acoustic signal based on the complete sound pressure sampling sequence of each acoustic signal and the reference sequence of each acoustic signal. The leakage source image generation module is used to calculate the spatial coordinates of the leakage source in the target area based on each time delay value; perform coordinate mapping according to the preset rotation angle and preset translation vector, and the spatial coordinates of the leakage source in the target area, to generate the target pixel coordinates of the leakage source in the target area; and perform image rendering with the target pixel coordinates of the leakage source in the target area as the center to generate the leakage source image in the target area. The image fusion module is used to fuse the leak source image and the visible light image to generate a leak source fused image.

[0014] Based on the above method embodiments, the present invention provides corresponding electronic device embodiments.

[0015] An embodiment of the present invention provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the leakage source localization method based on acoustic imaging and visible light image fusion as described in any of the above-described method embodiments.

[0016] Based on the above method embodiments, the present invention provides corresponding storage medium embodiments.

[0017] One embodiment of the present invention provides a storage medium storing a computer program thereon, wherein, when the computer program is running, it controls the device where the storage medium is located to execute any of the above-described method embodiments of the method for locating leakage sources based on acoustic imaging and visible light image fusion.

[0018] Compared with the prior art, the present invention has the following beneficial effects: This invention provides a method, apparatus, electronic device, and storage medium for locating leakage sources based on the fusion of acoustic imaging and visible light images. The method acquires acoustic signals and visible light images collected by multiple sensors within a target area; samples the sound pressure value of each acoustic signal to form a complete sound pressure sampling sequence; selects a sound pressure sampling point located in a preset central interval from the complete sound pressure sampling sequence to construct a reference sampling sequence; calculates the time delay value between the acoustic signals based on the complete sound pressure sampling sequence and the reference sampling sequence; determines the spatial coordinates of the leakage source using the time delay value; maps the spatial coordinates according to a preset rotation angle and translation vector to obtain the pixel coordinates of the leakage source in the image; renders the image centered on the pixel coordinates of the leakage source to generate a leakage source image, and fuses it with the visible light image to obtain a fused leakage source image.

[0019] This invention generates a shorter reference sequence by selecting sound pressure sampling points located in a preset middle interval from a complete sound pressure sampling sequence. This selection operation actively discards noisy head and tail edge data. Furthermore, when calculating the time delay, this invention uses the reference sequence to perform cross-correlation calculations with the complete sound pressure sampling sequence. Compared to the prior art method of calculating with one complete sound pressure sampling sequence and another, this method significantly reduces the overall computational load. This invention solves the problem of low leakage source localization efficiency caused by the huge computational load in the prior art without sacrificing positioning accuracy. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a method for locating leakage sources based on the fusion of acoustic imaging and visible light images, provided in an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of a leakage source localization device based on the fusion of acoustic imaging and visible light images, provided in an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] like Figure 1 As shown, to address the problem of low efficiency in locating leakage sources in existing technologies, an embodiment of the present invention provides a leakage source localization method based on the fusion of acoustic imaging and visible light images, comprising at least the following steps: Step S1: Acquire multiple acoustic signals of the target area and a visible light image of the target area; wherein each acoustic signal is acquired by a sensor in the sensor array and corresponds to a sensor in the sensor array.

[0024] In one specific embodiment, the target area refers to the physical space to be detected, which is jointly aligned by the sensor array and the camera module. The visible light image refers to an image frame acquired in real time by the camera module, used to characterize the real environmental background of the target area.

[0025] It should be noted that when a leak occurs in the target area, the leak source generates an acoustic event. This acoustic event propagates outward from the leak source in the form of sound waves. Multiple sensors in the sensor array are positioned at different physical coordinates. When the sound waves propagate to the sensor array, they arrive at the multiple sensors at different times. Each sensor in the array (e.g., a microphone), upon receiving the sound waves, converts them into its own analog electrical signal. These analog electrical signals, generated by different sensors and not yet sampled, constitute the acquired "multiple acoustic signals." Since these "multiple acoustic signals" essentially originate from the same acoustic event, they are highly correlated in terms of waveform characteristics, but exhibit a phase difference on the time axis due to the difference in the sound wave propagation path.

[0026] To achieve the fusion of the leak source image and the visible light image, the acquisition of the visible light image needs to meet specific requirements: First, the acquisition of the visible light image needs to be time-synchronized with the acquisition of the multiple acoustic signals to ensure the real-time performance and accuracy of the subsequent fused image. Second, there must be a fixed spatial correspondence between the optical coordinate system of the visible light image and the acoustic coordinate system of the sensor array. This fixed spatial correspondence is the necessary basis for the subsequent coordinate mapping (i.e., registration) step. Mathematically, this fixed spatial correspondence is defined as an affine transformation, characterized by a preset rotation angle and a preset translation vector. Both the preset rotation angle and the preset translation vector are pre-determined and stored through a pre-shipment calibration process (e.g., center calibration).

[0027] Furthermore, to improve the signal-to-noise ratio in subsequent processing, the acquired raw acoustic signal is processed by a bandpass filter before entering the sampling stage in step S2. The bandpass filter can be an eighth-order Chebyshev bandpass FIR filter, and its transfer function can be expressed as: In the formula, Let be the transfer function of the filter; The order of the filter; For time indexing; These are the filter coefficients.

[0028] By performing this step, a one-to-one correspondence of multi-channel acoustic data and real-time optical background images is prepared for subsequent time delay calculations and image fusion.

[0029] Step S2: Sample the sound pressure value of each acoustic signal to generate sound pressure sampling points for each acoustic signal; generate a complete sound pressure sampling sequence for each acoustic signal based on the sound pressure sampling points for each acoustic signal.

[0030] Specifically, sampling operation refers to sampling at a preset sampling frequency ( The filtered acoustic signal undergoes analog-to-digital conversion. The sampling frequency ( ) and the time interval between sampling points ( The following relationship exists between them: In the formula, The time interval between sampling points; The sampling frequency.

[0031] The sound pressure sampling points are the time indices of the analog-to-digital conversion. The discrete sound pressure values ​​obtained at the location. The complete sound pressure sampling sequence refers to the sequence formed by arranging all sound pressure sampling points within a data frame in chronological order.

[0032] By performing this step, continuous analog acoustic signals are converted into discrete digital sequences, providing the necessary data foundation for subsequent digital processing, such as generating reference sequences and calculating time delays.

[0033] Step S3: Select the sound pressure sampling point located in the preset middle interval from the complete sound pressure sampling sequence of each acoustic signal as the reference sampling point of each acoustic signal; generate the reference sequence of each acoustic signal based on the reference sampling point of each acoustic signal.

[0034] In a preferred embodiment, the preset middle interval refers to the set of continuous sampling points remaining in the complete sound pressure sampling sequence after excluding some sampling points at the beginning and end of the sequence. The reference sampling points are the sampling points within the middle interval. The reference sequence is a sequence formed by arranging the reference sampling points in their original chronological order. The reference sequence is shorter in data length than the complete sound pressure sampling sequence. The reference sequence corresponds to the cross-correlation calculation formula. In sequence.

[0035] By performing this step, the head and tail sampling points can be intentionally discarded, generating a shorter reference sequence. This reference sequence is used for cross-correlation calculations in subsequent steps. Replacing the complete sound pressure sampling sequence with the reference sequence significantly reduces the overall computational load of data processing, thus solving the inefficiency problem caused by processing all sampled data.

[0036] Step S4: Calculate the time delay between each acoustic signal based on the complete sound pressure sampling sequence of each acoustic signal and the reference sequence of each acoustic signal.

[0037] In a preferred embodiment, calculating the time delay value between each acoustic signal based on the complete sound pressure sampling sequence of each acoustic signal and a reference sequence of each acoustic signal includes: For each acoustic signal, the current acoustic signal is taken as the first acoustic signal, and the other acoustic signals besides the current acoustic signal are taken as the second acoustic signals. For each second acoustic signal, the reference sequence of the first acoustic signal is cross-correlated with the complete sound pressure sampling sequence of the current second acoustic signal to generate a cross-correlation function between the first acoustic signal and the current second acoustic signal; from the cross-correlation function between the first acoustic signal and the current second acoustic signal, the time offset when the cross-correlation function value is the largest is selected as the time delay value between the first acoustic signal and the current second acoustic signal.

[0038] In a preferred embodiment, the cross-correlation function between the first acoustic signal and the current second acoustic signal is generated by performing a cross-correlation calculation on the reference sequence of the first acoustic signal and the complete sound pressure sampling sequence of the current second acoustic signal using the following formula: In the formula, For the first The acoustic signal and the first The cross-correlation function of the acoustic signals; This is the time offset; For time indexing; For the first The reference sequence of acoustic signals at time index The sound pressure level at that location; For the first The complete sound pressure level sampling sequence of the acoustic signal, after offset, is indexed in time. The sound pressure level at that location.

[0039] It should be noted that this formula is a preferred implementation for calculating the cross-correlation function. The physical meaning of cross-correlation calculation lies in quantizing a signal (the complete sound pressure sampling sequence of the current second acoustic signal). After different time offsets After translation, it is compared with another signal (the reference sequence of the first acoustic signal). The degree of similarity between the two. Cross-correlation function values. The higher the value, the greater the time offset between the two signals. The stronger the correlation, the better. In this embodiment of the invention, cross-correlation calculation uses a reference sequence with a shorter data length. With complete sound pressure sampling sequence This method significantly reduces computational load while maintaining the correlation peak position, thus solving the problem of inefficient leakage source localization. It involves traversing different time offsets. By calculating the corresponding cross-correlation function value, one can find the value that makes the cross-correlation function value... To reach the maximum time offset.

[0040] By performing this step, asymmetric cross-correlation calculations can be performed using a reference sequence with less data (i.e., the reference sequence of the first acoustic signal) and the complete sound pressure sampling sequence (i.e., the complete sound pressure sampling sequence of the current second acoustic signal), which significantly reduces the amount of computation while ensuring positioning accuracy.

[0041] Step S5: Calculate the spatial coordinates of the leakage source in the target area based on each time delay value.

[0042] In a preferred embodiment, the spatial coordinates of the leakage source in the target area are calculated based on each time delay value using the following formula: In the formula, The spatial coordinates of the leakage source in the target area; For the first The spatial coordinates of the sensor corresponding to each acoustic signal; For the first The spatial coordinates of the sensor corresponding to each acoustic signal; For the first The acoustic signal and the first The time delay value of an acoustic signal; The speed of sound.

[0043] It should be noted that the same acoustic signal emitted by the leak source reaches two different sensors. and The difference in path length (left side of the formula) must be equal to the speed of sound. Multiply by the time delay values ​​measured by the two sensors ( (Right side of the formula). Mathematically, this formula defines a sensor. and sensors A hyperboloid with focal point. This is achieved by using multiple sets of time delay values ​​( Substituting these equations into the above formula, a system of nonlinear equations can be established. The unique solution to this system of equations (i.e., the intersection of all hyperboloids) is the spatial coordinate of the leakage source in three-dimensional space. .

[0044] Step S6: Based on the preset rotation angle, the preset translation vector, and the spatial coordinates of the leakage source in the target area, perform coordinate mapping to generate the target pixel coordinates of the leakage source in the target area.

[0045] In a preferred embodiment, the step of generating target pixel coordinates of the leakage source in the target region by performing coordinate mapping based on a preset rotation angle, a preset translation vector, and the spatial coordinates of the leakage source in the target region includes: Two-dimensional coordinate components are extracted from the spatial coordinates of the leakage source in the target area; The two-dimensional coordinate components are rotated according to a preset rotation angle to generate rotated coordinates; The translated coordinates are generated by performing a translation operation on the rotated coordinates according to the preset translation vector. The translated coordinates are used to determine the target pixel coordinates of the leakage source in the target area.

[0046] Specifically, extracting the two-dimensional coordinate components refers to extracting the two-dimensional coordinate components from the three-dimensional spatial coordinates. Extract from Two components, ignoring depth Components are used to achieve dimensionality reduction mapping from three-dimensional acoustic space to two-dimensional optical image plane.

[0047] The rotation and translation operations can be mathematically represented as an affine transformation. In a specific embodiment, the coordinate mapping can be achieved using the following general formula: In the formula, The target pixel coordinates; The preset rotation angle; This is a preset translation vector; These are two-dimensional coordinate components.

[0048] Furthermore, the preset rotation angle and preset translation vector are both pre-determined and stored through a calibration process (such as center calibration) before shipment. The calibration process ensures accurate spatial registration between the acoustic coordinate system of the sensor array and the optical coordinate system of the camera module.

[0049] For example, the method for obtaining the preset rotation angle and preset translation vector in step S6 will be explained. These parameters are pre-determined and stored through a one-time calibration step before the equipment leaves the factory or during installation.

[0050] Preset rotation angle The angle is fixed by the physical mounting angle of the sensor array and camera module on the device. The physical mounting angle is a known geometric parameter.

[0051] Preset translation vector This is specifically determined through a central calibration procedure. The central calibration procedure may include: First, a calibration mark (e.g., a bullseye or the center point of a checkerboard) is precisely placed at the geometric center of the sensor array. The coordinates of this geometric center in the acoustic coordinate system of the sensor array are known reference coordinates. (For example ).

[0052] Next, the camera module acquires a visible light image containing calibration marks. The pixel positions of the calibration marks in the optical coordinate system of the visible light image are measured. .

[0053] Based on the known rotation angle and the measured pixel positions and reference coordinates The preset translation vector is generated by calculating the following affine transformation relation. : In the formula, The measured pixel position; For the preset rotation angle The determined rotation matrix; The reference coordinates are known. Let be the preset translation vector to be solved.

[0054] exist In the case of translation vector That is equal to the measured pixel position .

[0055] By performing this step, the coordinates of the three-dimensional sound source in the physical world were successfully converted into two-dimensional pixel coordinates on the visible light image plane, providing accurate position input for image rendering in subsequent steps.

[0056] Step S7: Using the target pixel coordinates of the leakage source in the target area as the center, perform image rendering to generate an image of the leakage source in the target area.

[0057] In a preferred embodiment, the step of rendering the image to generate an image of the leakage source in the target area, centered on the target pixel coordinates of the leakage source in the target area, includes: An image matrix of a preset size is constructed with the target pixel coordinates of the leakage source in the target area as the center; Set the center pixel of the image matrix to a preset maximum intensity value, and set the boundary pixels of the image matrix to a preset minimum intensity value; Based on the maximum intensity value and the minimum intensity value, interpolation calculations are performed on the remaining pixels within the image matrix to generate an intensity distribution map; Based on the intensity distribution map, pseudo-color mapping is performed on the image matrix to generate a leakage source image of the target area.

[0058] Specifically, the target pixel coordinates are received in step S6, and a two-dimensional image matrix defined by a preset size is constructed in memory using these coordinates as the geometric center. Then, boundary conditions for interpolation calculations are set for the image matrix: the intensity value of the center pixel (i.e., the target pixel coordinates) is set to a preset maximum intensity value, and the intensity values ​​of all outermost boundary pixels are set to preset minimum intensity values. After setting the boundary conditions, interpolation calculations are performed on all remaining pixels within the image matrix. The interpolation calculations calculate a smooth transition intensity value based on the relative position of each internal pixel to the center (maximum intensity value) and the boundary (minimum intensity value). The product of the interpolation calculation is the intensity distribution map. Finally, pseudo-color mapping is performed on the intensity distribution map (a numerical matrix containing all interpolation results). This pseudo-color mapping uses a preset color lookup table (LUT) to convert each numerical intensity value in the intensity distribution map into a corresponding RGB color pixel value (e.g., mapping the maximum intensity to a warm tone and the minimum intensity to a cool tone), thereby generating the leak source image.

[0059] Step S8: Fuse the leak source image and the visible light image to generate a fused leak source image.

[0060] In a preferred embodiment, fusing the leak source image and the visible light image to generate a fused leak source image includes: Set preset transparency parameters for the leak source image; Based on the transparency parameter, the image of the leakage source is superimposed on the corresponding pixels of the visible light image to generate an intermediate fused image; The intermediate fused image is subjected to contrast enhancement processing to generate a leak source fused image.

[0061] Specifically, a preset transparency parameter is set for the leak source image. This transparency parameter (also known as the alpha value in image processing) is a preset value that defines the opacity of the leak source image. This value is used to balance the visual weight of the leak source image (colored light spot) and the visible light image (background) in subsequent overlay operations. Then, based on the transparency parameter, the leak source image is overlaid on the corresponding pixels of the visible light image to generate an intermediate blended image. This overlay operation is a pixel-level fusion operation, traversing every pixel in the region where the leak source image is located, and weighted averaging the pixel values ​​of the leak source image and the visible light image at the same coordinates according to the transparency parameter. This weighted average constitutes the intermediate blended image. Finally, contrast enhancement processing is applied to the intermediate blended image to generate a leak source fused image. This contrast enhancement processing is an image post-processing operation (e.g., histogram equalization or non-linear enhancement), which increases the contrast between the leak source image and the visible light image background by stretching the pixel intensity distribution range of the intermediate blended image, making the final leak source fused image visually clearer and easier to identify.

[0062] Based on the above method embodiments, the present invention provides corresponding apparatus embodiments.

[0063] like Figure 2 As shown, an embodiment of the present invention provides a leakage source localization device based on acoustic imaging and visible light image fusion, including: a data acquisition module, a sequence generation module, a time delay calculation module, a leakage image generation module and an image fusion module; The data acquisition module is used to acquire multiple acoustic signals of the target area and a visible light image of the target area; wherein each acoustic signal is acquired by a sensor in the sensor array and corresponds to a sensor in the sensor array; The sequence generation module is used to sample the sound pressure values ​​of each acoustic signal to generate sound pressure sampling points for each acoustic signal; generate a complete sound pressure sampling sequence for each acoustic signal based on the sound pressure sampling points; select sound pressure sampling points located in a preset middle interval from the complete sound pressure sampling sequences of each acoustic signal as reference sampling points for each acoustic signal; and generate a reference sequence for each acoustic signal based on the reference sampling points for each acoustic signal. The time delay calculation module is used to calculate the time delay value between each acoustic signal based on the complete sound pressure sampling sequence of each acoustic signal and the reference sequence of each acoustic signal. The leakage source image generation module is used to calculate the spatial coordinates of the leakage source in the target area based on each time delay value; perform coordinate mapping according to the preset rotation angle and preset translation vector, and the spatial coordinates of the leakage source in the target area, to generate the target pixel coordinates of the leakage source in the target area; and perform image rendering with the target pixel coordinates of the leakage source in the target area as the center to generate the leakage source image in the target area. The image fusion module is used to fuse the leak source image and the visible light image to generate a leak source fused image.

[0064] It should be noted that the embodiments of the device described above correspond to the embodiments of the present invention described above, and can realize the leakage source localization method based on acoustic imaging and visible light image fusion as described in any one of the above embodiments of the present invention. Furthermore, the embodiments of the device described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without creative effort.

[0065] Based on the above-described method embodiments of the present invention, a corresponding embodiment of an electronic device is provided.

[0066] An embodiment of the present invention provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the leakage source localization method based on acoustic imaging and visible light image fusion as described in any one of the present invention, or the processor executes the computer program to implement the functions of each module in the above-described device embodiments.

[0067] For example, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the terminal device.

[0068] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0069] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0070] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD card), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0071] Based on the above method embodiments, the present invention provides corresponding storage medium embodiments; Another embodiment of the present invention provides a storage medium including a stored computer program, wherein, when the computer program is running, it controls the device where the storage medium is located to execute any of the above-described leakage source localization methods based on acoustic imaging and visible light image fusion of the present invention.

[0072] The aforementioned storage medium is a computer-readable storage medium, and the computer program includes computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0073] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0074] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for locating leakage sources based on the fusion of acoustic imaging and visible light images, characterized in that, include: Multiple acoustic signals of the target area and a visible light image of the target area are acquired; wherein each acoustic signal is acquired by a sensor in the sensor array and corresponds to a sensor in the sensor array; The sound pressure value of each acoustic signal is sampled to generate sound pressure sampling points for each acoustic signal; based on the sound pressure sampling points of each acoustic signal, a complete sound pressure sampling sequence for each acoustic signal is generated. From the complete sound pressure sampling sequence of each acoustic signal, select the sound pressure sampling point located in the preset middle interval as the reference sampling point of each acoustic signal; generate the reference sequence of each acoustic signal based on the reference sampling point of each acoustic signal. Based on the complete sound pressure sampling sequence of each acoustic signal and the reference sequence of each acoustic signal, the time delay value between each acoustic signal is calculated; Based on each time delay value, the spatial coordinates of the leakage source in the target area are calculated; Based on the preset rotation angle and preset translation vector, as well as the spatial coordinates of the leakage source in the target area, coordinate mapping is performed to generate the target pixel coordinates of the leakage source in the target area. Using the target pixel coordinates of the leakage source in the target area as the center, image rendering is performed to generate an image of the leakage source in the target area; The leak source image and the visible light image are fused to generate a fused leak source image.

2. The leakage source localization method based on acoustic imaging and visible light image fusion as described in claim 1, characterized in that, The step of calculating the time delay between acoustic signals based on the complete sound pressure sampling sequence of each acoustic signal and the reference sequence of each acoustic signal includes: For each acoustic signal, the current acoustic signal is taken as the first acoustic signal, and the other acoustic signals besides the current acoustic signal are taken as the second acoustic signals. For each second acoustic signal, the reference sequence of the first acoustic signal is cross-correlated with the complete sound pressure sampling sequence of the current second acoustic signal to generate a cross-correlation function between the first acoustic signal and the current second acoustic signal; from the cross-correlation function between the first acoustic signal and the current second acoustic signal, the time offset when the cross-correlation function value is the largest is selected as the time delay value between the first acoustic signal and the current second acoustic signal.

3. The leakage source localization method based on acoustic imaging and visible light image fusion as described in claim 2, characterized in that, The cross-correlation function between the first acoustic signal and the current second acoustic signal is generated by performing a cross-correlation calculation on the reference sequence of the first acoustic signal and the complete sound pressure sampling sequence of the current second acoustic signal using the following formula: In the formula, For the first The acoustic signal and the first The cross-correlation function of the acoustic signals; This is the time offset; For time indexing; For the first The reference sequence of acoustic signals at time index The sound pressure level at that location; For the first The complete sound pressure level sampling sequence of the acoustic signal, after offset, is indexed in time. The sound pressure level at that location.

4. The leakage source localization method based on acoustic imaging and visible light image fusion as described in claim 3, characterized in that, The spatial coordinates of the leakage source in the target area are calculated using the following formula based on each time delay value: In the formula, The spatial coordinates of the leakage source in the target area; For the first The spatial coordinates of the sensor corresponding to each acoustic signal; For the first The spatial coordinates of the sensor corresponding to each acoustic signal; For the first The acoustic signal and the first The time delay value of an acoustic signal; The speed of sound.

5. The leakage source localization method based on acoustic imaging and visible light image fusion as described in claim 4, characterized in that, The step of generating target pixel coordinates of the leakage source in the target area by performing coordinate mapping based on a preset rotation angle, a preset translation vector, and the spatial coordinates of the leakage source in the target area includes: Two-dimensional coordinate components are extracted from the spatial coordinates of the leakage source in the target area; The two-dimensional coordinate components are rotated according to a preset rotation angle to generate rotated coordinates; The translated coordinates are generated by performing a translation operation on the rotated coordinates according to the preset translation vector. The translated coordinates are used to determine the target pixel coordinates of the leakage source in the target area.

6. The leakage source localization method based on acoustic imaging and visible light image fusion as described in claim 5, characterized in that, The step of rendering an image of the leakage source in the target area, centered on the target pixel coordinates of the leakage source, includes: An image matrix of a preset size is constructed with the target pixel coordinates of the leakage source in the target area as the center; Set the center pixel of the image matrix to a preset maximum intensity value, and set the boundary pixels of the image matrix to a preset minimum intensity value; Based on the maximum intensity value and the minimum intensity value, interpolation calculations are performed on the remaining pixels within the image matrix to generate an intensity distribution map; Based on the intensity distribution map, pseudo-color mapping is performed on the image matrix to generate a leakage source image of the target area.

7. The leakage source localization method based on acoustic imaging and visible light image fusion as described in claim 6, characterized in that, The step of fusing the leak source image and the visible light image to generate a fused leak source image includes: Set preset transparency parameters for the leak source image; Based on the transparency parameter, the image of the leakage source is superimposed on the corresponding pixels of the visible light image to generate an intermediate fused image; The intermediate fused image is subjected to contrast enhancement processing to generate a leak source fused image.

8. A leakage source localization device based on acoustic imaging and visible light image fusion, characterized in that, include: Data acquisition module, sequence generation module, delay calculation module, leaked image generation module, and image fusion module; The data acquisition module is used to acquire multiple acoustic signals of the target area and a visible light image of the target area; wherein each acoustic signal is acquired by a sensor in the sensor array and corresponds to a sensor in the sensor array; The sequence generation module is used to sample the sound pressure values ​​of each acoustic signal to generate sound pressure sampling points for each acoustic signal; generate a complete sound pressure sampling sequence for each acoustic signal based on the sound pressure sampling points; select sound pressure sampling points located in a preset middle interval from the complete sound pressure sampling sequences of each acoustic signal as reference sampling points for each acoustic signal; and generate a reference sequence for each acoustic signal based on the reference sampling points for each acoustic signal. The time delay calculation module is used to calculate the time delay value between each acoustic signal based on the complete sound pressure sampling sequence of each acoustic signal and the reference sequence of each acoustic signal. The leakage source image generation module is used to calculate the spatial coordinates of the leakage source in the target area based on each time delay value; perform coordinate mapping according to the preset rotation angle and preset translation vector, and the spatial coordinates of the leakage source in the target area, to generate the target pixel coordinates of the leakage source in the target area; and perform image rendering with the target pixel coordinates of the leakage source in the target area as the center to generate the leakage source image in the target area. The image fusion module is used to fuse the leak source image and the visible light image to generate a leak source fused image.

9. An electronic device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the leak source localization method based on acoustic imaging and visible light image fusion as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the storage medium is located to perform the leak source localization method based on acoustic imaging and visible light image fusion as described in any one of claims 1 to 7.

Images