Image processing method and device, computer device, storage medium and product

Abstract

This application relates to the field of industrial equipment fault diagnosis technology, and in particular to an image processing method, apparatus, computer equipment, storage medium, and product. The method includes: acquiring acoustic and infrared signals corresponding to a target device; performing coupled imaging based on the acoustic and infrared signals to obtain a coupled image of the target device; and reconstructing and enhancing the coupled image based on an acoustic feature dictionary corresponding to the acoustic signal and a thermal radiation feature dictionary corresponding to the infrared signal to obtain a target image of the target device. This application ensures that the target image can more accurately reflect the sound source and temperature conditions of the target device, enabling the target image to accurately reflect the operating status of the target device in complex environments.

Description

Technical Field

[0001] This application relates to the field of industrial equipment fault diagnosis technology, and in particular to an image processing method, apparatus, computer equipment, storage medium and product. Background Technology

[0002] An acoustic imager (also known as an acoustic camera or acoustic phaser) is an industrial inspection device based on microphone arrays and beamforming algorithms. It locates sound sources by capturing phase differences in sound wave signals and displays the sound field distribution as a color cloud map. Its core components include a microphone array, a signal processing unit, and a display unit. It is primarily used in compressed gas leak detection, partial discharge location in power equipment, and abnormal noise diagnosis in mechanical equipment, covering industries such as petrochemicals, power grids, and rail transportation.

[0003] However, traditional acoustic imagers only locate sound sources through microphone arrays and cannot correlate with temperature anomalies (such as localized overheating associated with discharge), resulting in a high probability of missed detection when using acoustic imaging to detect equipment faults. Summary of the Invention

[0004] Therefore, it is necessary to provide an image processing method, apparatus, computer equipment, storage medium, and product that can reduce the probability of missed fault detection, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides an image processing method. The method includes:

[0006] Acquire the acoustic and infrared signals corresponding to the target device;

[0007] Coupled imaging is performed based on the acoustic signal and the infrared signal to obtain a coupled image corresponding to the target device;

[0008] The coupled image is reconstructed and enhanced based on the acoustic feature dictionary corresponding to the acoustic wave signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain the target image corresponding to the target device.

[0009] In one embodiment, the step of reconstructing and enhancing the coupled image based on the acoustic feature dictionary corresponding to the acoustic signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain the target image corresponding to the target device includes:

[0010] Based on the acoustic feature dictionary and the thermal radiation feature dictionary, an overcomplete dictionary is constructed;

[0011] The coupled image is reconstructed and enhanced based on the overcomplete dictionary to obtain the target image corresponding to the target device.

[0012] In one embodiment, the step of reconstructing and enhancing the coupled image based on the overcomplete dictionary to obtain the target image corresponding to the target device includes:

[0013] Determine the sparse coefficients corresponding to the coupled image;

[0014] Based on the sparse coefficients and the overcomplete dictionary, construct the target image corresponding to the target device.

[0015] In one embodiment, constructing an overcomplete dictionary based on the acoustic feature dictionary and the thermal radiation feature dictionary includes:

[0016] The acoustic signal is filtered to obtain the processed acoustic signal;

[0017] The infrared signal is filtered to obtain the processed infrared signal;

[0018] An overcomplete dictionary is constructed based on the acoustic feature dictionary corresponding to the processed sound wave signal and the thermal radiation feature dictionary corresponding to the processed infrared signal.

[0019] In one embodiment, the step of performing coupled imaging based on the acoustic signal and the infrared signal to obtain a coupled image corresponding to the target device includes:

[0020] The acoustic signal is subjected to source intensity analysis to obtain the acoustic power corresponding to the acoustic signal;

[0021] Temperature field inversion is performed on the infrared signal to obtain the temperature value corresponding to the infrared signal;

[0022] Coupled imaging is performed based on the acoustic power and the temperature value to obtain a coupled image corresponding to the target device.

[0023] In one embodiment, the method further includes:

[0024] The abnormal coordinates contained in the target image are identified by performing anomaly coordinate identification based on a preset acoustic power threshold and a preset temperature threshold.

[0025] Secondly, this application also provides an image processing apparatus. The apparatus includes:

[0026] The acquisition module is used to acquire the acoustic and infrared signals corresponding to the target device.

[0027] An imaging module is used to perform coupled imaging based on the acoustic signal and the infrared signal to obtain a coupled image corresponding to the target device;

[0028] An enhancement module is used to reconstruct and enhance the coupled image based on the acoustic feature dictionary corresponding to the acoustic wave signal and the thermal radiation feature dictionary corresponding to the infrared signal, so as to obtain the target image corresponding to the target device.

[0029] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0030] Acquire the acoustic and infrared signals corresponding to the target device;

[0031] Coupled imaging is performed based on the acoustic signal and the infrared signal to obtain a coupled image corresponding to the target device;

[0032] The coupled image is reconstructed and enhanced based on the acoustic feature dictionary corresponding to the acoustic wave signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain the target image corresponding to the target device.

[0033] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0034] Acquire the acoustic and infrared signals corresponding to the target device;

[0035] Coupled imaging is performed based on the acoustic signal and the infrared signal to obtain a coupled image corresponding to the target device;

[0036] The coupled image is reconstructed and enhanced based on the acoustic feature dictionary corresponding to the acoustic wave signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain the target image corresponding to the target device.

[0037] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0038] Acquire the acoustic and infrared signals corresponding to the target device;

[0039] Coupled imaging is performed based on the acoustic signal and the infrared signal to obtain a coupled image corresponding to the target device;

[0040] The coupled image is reconstructed and enhanced based on the acoustic feature dictionary corresponding to the acoustic wave signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain the target image corresponding to the target device.

[0041] The aforementioned image processing method, apparatus, computer equipment, storage medium, and product acquire acoustic and infrared signals corresponding to the target device; perform coupled imaging based on the acoustic and infrared signals to obtain a coupled image of the target device; and finally reconstruct and enhance the coupled image based on the acoustic feature dictionary corresponding to the acoustic signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain a target image of the target device. As can be seen from the above, this application, in performing the image processing method, provides a data foundation for subsequent image processing by acquiring the acoustic and infrared signals corresponding to the target device. Furthermore, by performing coupled imaging with the acoustic and infrared signals, the infrared signal of the target device is incorporated into the acoustic imaging process, ensuring that the coupled image can effectively reflect the thermal radiation state of the target device, thus effectively reflecting any temperature anomalies. Further, by reconstructing and enhancing the coupled image, the target image can more accurately reflect the sound source and temperature conditions of the target device, enabling the target image to accurately reflect the operation of the target device in complex environments. Attached Figure Description

[0042] Figure 1 An application environment diagram of an image processing method provided in an embodiment of this application;

[0043] Figure 2 A schematic flowchart illustrating the first image processing method provided in this application embodiment;

[0044] Figure 3 A schematic flowchart illustrating the second image processing method provided in this application embodiment;

[0045] Figure 4 A schematic flowchart illustrating the third image processing method provided in this application embodiment;

[0046] Figure 5 A flowchart illustrating the fourth image processing method provided in this application embodiment;

[0047] Figure 6 A structural block diagram of an image processing apparatus provided in an embodiment of this application;

[0048] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0050] The image processing method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed in the cloud or on other network servers. The process involves acquiring the acoustic and infrared signals corresponding to the target device; performing coupled imaging based on the acoustic and infrared signals to obtain a coupled image of the target device; and finally reconstructing and enhancing the coupled image based on the acoustic feature dictionary corresponding to the acoustic signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain the target image of the target device. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0051] In one embodiment, such as Figure 2 As shown, an image processing method is provided, which is applied to... Figure 1 Taking server 104 as an example, the following steps are included:

[0052] S201, acquire the acoustic and infrared signals corresponding to the target device.

[0053] It should be noted that the acoustic signal of the target device can be acquired through a pre-deployed microphone array; the acoustic and infrared signals of the target device can be acquired through a pre-deployed infrared camera.

[0054] To further explain, in order to eliminate the spatiotemporal deviation between sound wave signals and infrared signals, the propagation time of the sound wave signal can be obtained first, and then the infrared signal can be synchronously corrected based on the sound wave propagation time.

[0055] Specifically, when calculating the propagation time of a sound wave signal, we can pre-assume the location of the sound source as (x0, y0), and then determine the distance from the microphone (i, j) to the sound source:

[0056] ;

[0057] Where the distance from the microphone (i,j) to the sound source is .

[0058] Then, the propagation time of the sound wave signal is determined according to the following formula:

[0059] ;

[0060] in, "c" refers to the propagation time of the sound wave signal, and "c" refers to the speed of sound.

[0061] Furthermore, according to Adjust the infrared frame number so that the k-th infrared image corresponds to the acoustic wave acquisition time t. k + The corrected infrared signal is:

[0062] I′(x,y,k)=I(x,y,k+round( / Δt)) (round() is for rounding to the nearest integer);

[0063] As an example, if the distance from a microphone to a sound source is 5.145 meters, =5.145 / 343≈0.015s, infrared frame interval Δt=0.01s, then the infrared frame needs to be delayed by 1 frame (0.015 / 0.01≈1.5→rounded to 1), that is: I′(x,y,k)=I(x,y,k+1).

[0064] S202, couples the acoustic signal and the infrared signal to obtain the coupled image of the target device.

[0065] It should be noted that when it is necessary to perform coupled imaging based on acoustic signals and infrared signals to obtain a coupled image of the target device, the following may be included: performing sound source intensity analysis on the acoustic signal to obtain the acoustic power corresponding to the acoustic signal; performing temperature field inversion on the infrared signal to obtain the temperature value corresponding to the infrared signal; and performing coupled imaging based on the acoustic power and temperature value to obtain a coupled image of the target device.

[0066] S203, the coupled image is reconstructed and enhanced based on the acoustic feature dictionary corresponding to the sound wave signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain the target image corresponding to the target device.

[0067] The acoustic feature dictionary contains atoms or basis vectors used to represent and describe various possible characteristic modes of acoustic signals. The thermal radiation feature dictionary contains atoms or basis vectors used to represent and describe various possible characteristic modes of infrared signals.

[0068] In one embodiment of this application, when it is necessary to reconstruct and enhance the coupled image based on the acoustic feature dictionary corresponding to the sound wave signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain the target image corresponding to the target device, the following may be included: constructing an overcomplete dictionary based on the acoustic feature dictionary and the thermal radiation feature dictionary; reconstructing and enhancing the coupled image based on the overcomplete dictionary to obtain the target image corresponding to the target device.

[0069] The aforementioned image processing method acquires the acoustic and infrared signals corresponding to the target device; performs coupled imaging based on the acoustic and infrared signals to obtain a coupled image of the target device; and finally reconstructs and enhances the coupled image based on the acoustic feature dictionary corresponding to the acoustic signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain the target image of the target device. As can be seen from the above, this application, in its image processing method, provides a data foundation for subsequent image processing by acquiring the acoustic and infrared signals corresponding to the target device. Furthermore, by performing coupled imaging based on the acoustic and infrared signals, the infrared signal of the target device is incorporated into the acoustic imaging process, ensuring that the coupled image can effectively reflect the thermal radiation state of the target device and thus effectively reflect any temperature anomalies. Moreover, by reconstructing and enhancing the coupled image, the target image more accurately reflects the sound source and temperature conditions of the target device, enabling it to accurately reflect the operating status of the target device in complex environments.

[0070] In one embodiment, such as Figure 3 As shown, when it is necessary to reconstruct and enhance the coupled image based on the acoustic feature dictionary corresponding to the sound wave signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain the target image corresponding to the target device, the following can be included:

[0071] S301, a complete dictionary was constructed based on the acoustic feature dictionary and the thermal radiation feature dictionary.

[0072] Among them, the overcomplete dictionary is the feature set obtained by combining and splicing the acoustic feature dictionary and the thermal radiation feature dictionary.

[0073] It should be noted that when an overcomplete dictionary needs to be constructed, the following can be included: filtering the acoustic signal to obtain the processed acoustic signal; filtering the infrared signal to obtain the processed infrared signal; and constructing an overcomplete dictionary based on the acoustic feature dictionary corresponding to the processed acoustic signal and the thermal radiation feature dictionary corresponding to the processed infrared signal.

[0074] In one embodiment of this application, when it is necessary to filter the acoustic signal and the infrared signal, noise estimation can be performed in advance to determine the acoustic noise corresponding to the acoustic signal. This refers to the variance of the acoustic signal in the region without a sound source; the infrared noise corresponding to the infrared signal. , which is the variance of the infrared signal in the low-temperature uniform region.

[0075] Specifically, when filtering of acoustic signals is required, variable step size LMS filtering can be used, with a step size factor of [missing information]. Dynamic adjustment:

[0076] ;

[0077] Where γ = 0.01 is a constant; Avoid having a denominator of zero.

[0078] When infrared signals need to be filtered, bilateral filtering with spatial weighting can be used. and grayscale weight for:

[0079] ; ;

[0080] in, For temperature standard deviation, follow change: , .

[0081] Furthermore, after filtering the acoustic signal and the infrared signal separately, they can be combined and filtered for output, as shown below:

[0082] ;

[0083] As an example, in a high-noise environment in a factory ( =100), =0.01 / (100+1e−6)≈0.0001, the filtering is smoother; if the infrared background noise is large ( =25), then =0.5×5=2.5, grayscale weight It attenuates faster and suppresses thermal reflection interference.

[0084] S302, the coupled image is reconstructed and enhanced according to the overcomplete dictionary to obtain the target image corresponding to the target device.

[0085] It should be noted that when it is necessary to reconstruct and enhance the coupled image based on the overcomplete dictionary to obtain the target image corresponding to the target device, the following may be included: determining the sparse coefficients corresponding to the coupled image; and constructing the target image corresponding to the target device based on the sparse coefficients and the overcomplete dictionary.

[0086] An overcomplete dictionary can be represented as: ;in, An acoustic feature dictionary; This is a dictionary of thermal radiation characteristics.

[0087] The sparsity coefficients of the coupled image can be solved using the following method:

[0088] ;

[0089] Where λ=0.1 is the regularization parameter; The L1 norm promotes sparsity.

[0090] The above image processing method constructs an overcomplete dictionary based on an acoustic feature dictionary and a thermal radiation feature dictionary; then, it reconstructs and enhances the coupled image based on the overcomplete dictionary to obtain the target image corresponding to the target device; by coupling imaging with acoustic signals and infrared signals, it integrates the infrared signal of the target device into the process of acoustic imaging of the target device, ensuring that the coupled image can effectively reflect the thermal radiation state of the target device, and enabling the coupled image to effectively reflect the temperature anomalies of the target device.

[0091] In one embodiment, such as Figure 4 As shown, when it is necessary to perform coupled imaging based on acoustic signals and infrared signals to obtain a coupled image of the target device, the following may be included:

[0092] S401 performs sound source intensity analysis on the sound wave signal to obtain the corresponding sound wave power.

[0093] It should be noted that the following calculation formula can be used when performing sound source intensity analysis on sound wave signals:

[0094]

[0095] in, This refers to the coordinates of the sound source; This refers to the sound wave power at the coordinates of the sound source; These are weighting coefficients used to handle sidelobe interference.

[0096] S402 performs temperature field inversion on the infrared signal to obtain the temperature value corresponding to the infrared signal.

[0097] It should be noted that when it is necessary to perform temperature field inversion on infrared signals, since the intensity and temperature of infrared signals satisfy Planck's law, the following formula can be used for temperature field inversion:

[0098] ;

[0099] Where a and b are calibration coefficients.

[0100] S403 performs coupled imaging based on acoustic power and temperature values ​​to obtain a coupled image of the target device.

[0101] It should be noted that when coupled imaging is required based on acoustic power and thermal power, the following calculation formula can be used:

[0102] ;

[0103] Where C represents a pixel in the coupled image; P refers to acoustic power; T refers to temperature; Q refers to thermal power, Q=kP, and k is the inherent coefficient of the target device.

[0104] The above image processing method uses acoustic power and temperature value to perform coupled imaging to obtain a coupled image of the target device. This method incorporates the infrared signal of the target device into the acoustic imaging process, ensuring that the coupled image can effectively reflect the thermal radiation state of the target device and thus effectively reflect any abnormal temperature conditions of the target device.

[0105] In one embodiment, after determining the target image corresponding to the target device, the following may also be included:

[0106] Anomaly coordinates are identified in the target image based on preset acoustic power thresholds and preset temperature thresholds, thus obtaining the anomaly coordinates contained in the target image.

[0107] In one embodiment of this application, for each candidate coordinate in the target image, it is verified whether the sound power value corresponding to the candidate mouth is greater than the sound power threshold and whether the temperature value is greater than the temperature threshold. Then, if the sound power value is greater than the sound power threshold or the temperature value is greater than the temperature threshold, the candidate coordinate is determined to be an abnormal coordinate.

[0108] Here, candidate coordinates refer to each coordinate in the target image that needs to be detected for anomalies.

[0109] The above image processing method identifies abnormal coordinates in the target image based on preset acoustic power thresholds and preset temperature thresholds, thereby obtaining the abnormal coordinates contained in the target image and realizing the identification operation for abnormal coordinates.

[0110] In one embodiment, such as Figure 5 As shown, when it is necessary to determine the target image corresponding to the target device, the following can be included:

[0111] S501, acquire the acoustic and infrared signals corresponding to the target device.

[0112] S502 performs sound source intensity analysis on the sound wave signal to obtain the sound wave power corresponding to the sound wave signal.

[0113] S503 performs temperature field inversion on the infrared signal to obtain the temperature value corresponding to the infrared signal.

[0114] S504 performs coupled imaging based on acoustic power and temperature values ​​to obtain a coupled image of the target device.

[0115] S505 filters the acoustic signal to obtain the processed acoustic signal.

[0116] S506 filters the infrared signal to obtain the processed infrared signal.

[0117] S507, based on the acoustic feature dictionary corresponding to the processed sound wave signal and the thermal radiation feature dictionary corresponding to the processed infrared signal, a complete dictionary is constructed.

[0118] S508, determine the sparse coefficients corresponding to the coupled image.

[0119] S509: Construct the target image corresponding to the target device based on the sparse coefficients and the overcomplete dictionary.

[0120] The aforementioned image processing method acquires the acoustic and infrared signals corresponding to the target device; performs coupled imaging based on the acoustic and infrared signals to obtain a coupled image of the target device; and finally reconstructs and enhances the coupled image based on the acoustic feature dictionary corresponding to the acoustic signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain the target image of the target device. As can be seen from the above, this application, in its image processing method, provides a data foundation for subsequent image processing by acquiring the acoustic and infrared signals corresponding to the target device. Furthermore, by performing coupled imaging based on the acoustic and infrared signals, the infrared signal of the target device is incorporated into the acoustic imaging process, ensuring that the coupled image can effectively reflect the thermal radiation state of the target device and thus effectively reflect any temperature anomalies. Moreover, by reconstructing and enhancing the coupled image, the target image more accurately reflects the sound source and temperature conditions of the target device, enabling it to accurately reflect the operating status of the target device in complex environments.

[0121] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0122] Based on the same inventive concept, this application also provides an image processing apparatus for implementing the image processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more image processing apparatus embodiments provided below can be found in the limitations of the image processing method described above, and will not be repeated here.

[0123] In one embodiment, such as Figure 6 As shown, an image processing apparatus is provided, comprising: an acquisition module 10, an imaging module 20, and an enhancement module 30, wherein:

[0124] The acquisition module 10 is used to acquire the acoustic signal and infrared signal corresponding to the target device.

[0125] The imaging module 20 is used to perform coupled imaging based on acoustic signals and infrared signals to obtain a coupled image of the target device.

[0126] The enhancement module 30 is used to reconstruct and enhance the coupled image based on the acoustic feature dictionary corresponding to the sound wave signal and the thermal radiation feature dictionary corresponding to the infrared signal, so as to obtain the target image corresponding to the target device.

[0127] In one embodiment, an overcomplete dictionary is constructed based on an acoustic feature dictionary and a thermal radiation feature dictionary;

[0128] The coupled image is reconstructed and enhanced based on an overcomplete dictionary to obtain the target image corresponding to the target device.

[0129] In one embodiment, the sparse coefficients corresponding to the coupled image are determined;

[0130] Based on the sparse coefficients and the overcomplete dictionary, construct the target image corresponding to the target device.

[0131] In one embodiment, the acoustic signal is filtered to obtain a processed acoustic signal;

[0132] The infrared signal is filtered to obtain the processed infrared signal;

[0133] An overcomplete dictionary is constructed based on the acoustic feature dictionary corresponding to the processed sound wave signal and the thermal radiation feature dictionary corresponding to the processed infrared signal.

[0134] In one embodiment, the sound source intensity of the sound wave signal is analyzed to obtain the sound wave power corresponding to the sound wave signal;

[0135] Temperature field inversion is performed on the infrared signal to obtain the corresponding temperature value;

[0136] Coupled imaging is performed based on acoustic power and temperature values ​​to obtain the coupled image corresponding to the target device.

[0137] In one embodiment, abnormal coordinates of the target image are identified based on a preset acoustic power threshold and a preset temperature threshold to obtain the abnormal coordinates contained in the target image.

[0138] The aforementioned image processing apparatus acquires the acoustic and infrared signals corresponding to the target device; performs coupled imaging based on the acoustic and infrared signals to obtain a coupled image of the target device; and finally reconstructs and enhances the coupled image based on the acoustic feature dictionary corresponding to the acoustic signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain a target image of the target device. As can be seen from the above, this application, in its image processing method, provides a data foundation for subsequent image processing by acquiring the acoustic and infrared signals corresponding to the target device. Furthermore, by performing coupled imaging based on the acoustic and infrared signals, the infrared signal of the target device is incorporated into the acoustic imaging process, ensuring that the coupled image can effectively reflect the thermal radiation state of the target device, thus effectively reflecting any temperature anomalies. Moreover, by reconstructing and enhancing the coupled image, the target image more accurately reflects the sound source and temperature conditions of the target device, enabling it to accurately reflect the operating status of the target device in complex environments.

[0139] Each module in the aforementioned image processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0140] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an image processing method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0141] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0142] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0143] Acquire the acoustic and infrared signals corresponding to the target device;

[0144] Coupled imaging is performed based on acoustic and infrared signals to obtain a coupled image of the target device.

[0145] The coupled image is reconstructed and enhanced based on the acoustic feature dictionary corresponding to the sound wave signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain the target image corresponding to the target device.

[0146] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0147] A comprehensive dictionary was constructed based on the acoustic feature dictionary and the thermal radiation feature dictionary;

[0148] The coupled image is reconstructed and enhanced based on an overcomplete dictionary to obtain the target image corresponding to the target device.

[0149] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0150] Determine the sparse coefficients corresponding to the coupled image;

[0151] Based on the sparse coefficients and the overcomplete dictionary, construct the target image corresponding to the target device.

[0152] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0153] The sound wave signal is filtered to obtain the processed sound wave signal;

[0154] The infrared signal is filtered to obtain the processed infrared signal;

[0155] An overcomplete dictionary is constructed based on the acoustic feature dictionary corresponding to the processed sound wave signal and the thermal radiation feature dictionary corresponding to the processed infrared signal.

[0156] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0157] By performing sound source intensity analysis on the sound wave signal, the corresponding sound wave power can be obtained;

[0158] Temperature field inversion is performed on the infrared signal to obtain the corresponding temperature value;

[0159] Coupled imaging is performed based on acoustic power and temperature values ​​to obtain the coupled image corresponding to the target device.

[0160] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0161] Anomaly coordinates are identified in the target image based on preset acoustic power thresholds and preset temperature thresholds, thus obtaining the anomaly coordinates contained in the target image.

[0162] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0163] Acquire the acoustic and infrared signals corresponding to the target device;

[0164] Coupled imaging is performed based on acoustic and infrared signals to obtain a coupled image of the target device.

[0165] The coupled image is reconstructed and enhanced based on the acoustic feature dictionary corresponding to the sound wave signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain the target image corresponding to the target device.

[0166] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0167] A comprehensive dictionary was constructed based on the acoustic feature dictionary and the thermal radiation feature dictionary;

[0168] The coupled image is reconstructed and enhanced based on an overcomplete dictionary to obtain the target image corresponding to the target device.

[0169] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0170] Determine the sparse coefficients corresponding to the coupled image;

[0171] Based on the sparse coefficients and the overcomplete dictionary, construct the target image corresponding to the target device.

[0172] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0173] The sound wave signal is filtered to obtain the processed sound wave signal;

[0174] The infrared signal is filtered to obtain the processed infrared signal;

[0175] An overcomplete dictionary is constructed based on the acoustic feature dictionary corresponding to the processed sound wave signal and the thermal radiation feature dictionary corresponding to the processed infrared signal.

[0176] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0177] By performing sound source intensity analysis on the sound wave signal, the corresponding sound wave power can be obtained;

[0178] Temperature field inversion is performed on the infrared signal to obtain the corresponding temperature value;

[0179] Coupled imaging is performed based on acoustic power and temperature values ​​to obtain the coupled image corresponding to the target device.

[0180] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0181] Anomaly coordinates are identified in the target image based on preset acoustic power thresholds and preset temperature thresholds, thus obtaining the anomaly coordinates contained in the target image.

[0182] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0183] Acquire the acoustic and infrared signals corresponding to the target device;

[0184] Coupled imaging is performed based on acoustic and infrared signals to obtain a coupled image of the target device.

[0185] The coupled image is reconstructed and enhanced based on the acoustic feature dictionary corresponding to the sound wave signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain the target image corresponding to the target device.

[0186] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0187] A comprehensive dictionary was constructed based on the acoustic feature dictionary and the thermal radiation feature dictionary;

[0188] The coupled image is reconstructed and enhanced based on an overcomplete dictionary to obtain the target image corresponding to the target device.

[0189] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0190] …

[0191] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0192] Determine the sparse coefficients corresponding to the coupled image;

[0193] Based on the sparse coefficients and the overcomplete dictionary, construct the target image corresponding to the target device.

[0194] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0195] The sound wave signal is filtered to obtain the processed sound wave signal;

[0196] The infrared signal is filtered to obtain the processed infrared signal;

[0197] An overcomplete dictionary is constructed based on the acoustic feature dictionary corresponding to the processed sound wave signal and the thermal radiation feature dictionary corresponding to the processed infrared signal.

[0198] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0199] By performing sound source intensity analysis on the sound wave signal, the corresponding sound wave power can be obtained;

[0200] Temperature field inversion is performed on the infrared signal to obtain the corresponding temperature value;

[0201] Coupled imaging is performed based on acoustic power and temperature values ​​to obtain the coupled image corresponding to the target device.

[0202] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0203] Anomaly coordinates are identified in the target image based on preset acoustic power thresholds and preset temperature thresholds, thus obtaining the anomaly coordinates contained in the target image.

[0204] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0205] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0206] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0207] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: Acquire the acoustic and infrared signals corresponding to the target device; Coupled imaging is performed based on the acoustic signal and the infrared signal to obtain a coupled image corresponding to the target device; The coupled image is reconstructed and enhanced based on the acoustic feature dictionary corresponding to the acoustic wave signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain the target image corresponding to the target device.

2. The method according to claim 1, characterized in that, The step of reconstructing and enhancing the coupled image based on the acoustic feature dictionary corresponding to the acoustic signal and the thermal radiation feature dictionary corresponding to the infrared signal to obtain the target image corresponding to the target device includes: Based on the acoustic feature dictionary and the thermal radiation feature dictionary, an overcomplete dictionary is constructed; The coupled image is reconstructed and enhanced based on the overcomplete dictionary to obtain the target image corresponding to the target device.

3. The method according to claim 2, characterized in that, The step of reconstructing and enhancing the coupled image based on the overcomplete dictionary to obtain the target image corresponding to the target device includes: Determine the sparse coefficients corresponding to the coupled image; Based on the sparse coefficients and the overcomplete dictionary, construct the target image corresponding to the target device.

4. The method according to claim 2, characterized in that, The construction of a complete dictionary based on the acoustic feature dictionary and the thermal radiation feature dictionary includes: The acoustic signal is filtered to obtain the processed acoustic signal; The infrared signal is filtered to obtain the processed infrared signal; An overcomplete dictionary is constructed based on the acoustic feature dictionary corresponding to the processed sound wave signal and the thermal radiation feature dictionary corresponding to the processed infrared signal.

5. The method according to any one of claims 1-4, characterized in that, The step of performing coupled imaging based on the acoustic signal and the infrared signal to obtain a coupled image corresponding to the target device includes: The acoustic signal is subjected to source intensity analysis to obtain the acoustic power corresponding to the acoustic signal; Temperature field inversion is performed on the infrared signal to obtain the temperature value corresponding to the infrared signal; Coupled imaging is performed based on the acoustic power and the temperature value to obtain a coupled image corresponding to the target device.

6. The method according to any one of claims 1-4, characterized in that, The method further includes: The abnormal coordinates contained in the target image are identified by performing anomaly coordinate identification based on a preset acoustic power threshold and a preset temperature threshold.

7. An image processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire the acoustic and infrared signals corresponding to the target device. An imaging module is used to perform coupled imaging based on the acoustic signal and the infrared signal to obtain a coupled image corresponding to the target device; An enhancement module is used to reconstruct and enhance the coupled image based on the acoustic feature dictionary corresponding to the acoustic wave signal and the thermal radiation feature dictionary corresponding to the infrared signal, so as to obtain the target image corresponding to the target device.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images