Array multi-channel acoustic imaging device and abnormal sound source positioning display method

Abstract

The application discloses an array type multi-channel acoustic imaging device and an abnormal sound source positioning display method. The device comprises an acoustic imaging cooperation request module, which is used for sending an imaging cooperation request instruction and target information related to imaging to a background server, receiving returned information about whether to trigger a cooperation mechanism, and sending an environmental data acquisition instruction. An environmental data acquisition device is used for collecting environmental data. The acoustic imaging cooperation request module is also used for sending performance index data of the acoustic imaging device and the acquired environmental data to the background server, receiving returned master-slave imaging device information, generating an acoustic imaging cooperation processing module running instruction when it is determined to be a master imaging device, and acquiring autonomous imaging data of the cooperative acoustic imaging device for space-time alignment processing and data weighted fusion to obtain an abnormal sound source sound field image about the target to be imaged. The application can accurately image the target to be imaged in a complex scene.

Description

Technical Field

[0001] This application relates to the technical field of acoustic imaging, specifically to an array-type multi-channel acoustic imaging device and a method for locating and displaying abnormal sound sources. Background Technology

[0002] In the field of acoustic imaging technology, with the continuous development of industrial production, scientific research, and other fields, the demand for accurately acquiring and analyzing sound information is increasing. Acoustic imaging technology can visualize sound, helping people intuitively understand the distribution and propagation of sound, which is of great significance in fault detection, noise source localization, and building acoustic assessment. It not only improves work efficiency but also provides a powerful tool for solving complex acoustic problems, thus promoting technological progress in related fields.

[0003] In traditional acoustic imaging technology, conventional methods are typically employed to achieve acoustic imaging. A common approach is to use a single microphone array to acquire sound signals. Some existing technologies also utilize array microphones for acoustic imaging, and by combining them with other imaging techniques, such as thermal imaging and video recording, their application scope is further expanded. This allows for the simultaneous acquisition of sound, temperature, and visual image information. For example, patent application CN202510088516.4 discloses an array-type multi-channel acoustic imaging device that effectively locates abnormal sound sources and displays their temperatures in complex environments, improving the accuracy of location and temperature measurement. However, existing technologies primarily rely on the independent operation of a single device, lacking coordination with other devices. Furthermore, when facing different imaging scenarios, fixed algorithms and parameters are often used, failing to flexibly adjust according to changes in the scene. This makes it difficult to comprehensively and accurately reflect the actual situation of the target scene, resulting in less than ideal imaging effects.

[0004] In summary, how to provide an acoustic imaging device that considers multi-device collaborative positioning and accurately images the target in complex scenarios, and how to improve the accuracy of abnormal sound source localization while simultaneously acquiring temperature information are urgent technical problems that need to be solved. Summary of the Invention

[0005] In order to provide an acoustic imaging device that takes into account multi-device collaborative positioning and accurately images the target in complex scenarios, improves the accuracy of abnormal sound source localization and simultaneously acquires temperature information, this application provides an array-type multi-channel acoustic imaging device and an abnormal sound source localization and display method.

[0006] In a first aspect, this application provides an array-type multi-channel acoustic imaging device, including: an imaging display device, a main microphone array, a camera device and a thermal imaging device; it also includes: an environmental data acquisition device, an acoustic imaging collaboration request module and an acoustic imaging collaboration processing module that establish communication with the imaging display device;

[0007] The acoustic imaging collaboration request module is used to send an imaging collaboration request instruction, the location of the imaging display device, and / or the information of the target to be imaged to the backend server when the acoustic imaging device is started. This allows the backend server to determine, based on the scene type of the target to be imaged, whether the content sent by multiple acoustic field imaging devices meets the minimum number of collaborative devices and topology requirements matching the scene type within a preset range of the target to be imaged, thereby generating information to trigger the collaboration mechanism. The module also receives information from the backend server indicating whether the collaboration mechanism has been triggered, and sends an environmental data acquisition instruction to the environmental data acquisition device when the information to trigger the collaboration mechanism is received.

[0008] The environmental data acquisition device is used to acquire environmental data after receiving an environmental data acquisition command;

[0009] The acoustic imaging collaboration request module is also used to send the acquired environmental data and performance index data of the acoustic imaging device to the backend server, so that the backend server can determine the master and slave imaging devices in the collaborative acoustic imaging device according to the environmental data and performance index data transmitted by each acoustic imaging device and the distance to the target to be imaged calculated according to the location of the imaging display device and the target information; receive the master and slave imaging device information returned by the backend server, determine whether it is the master imaging device of the target to be imaged, generate an acoustic imaging collaboration processing module running instruction when it is determined to be the master imaging device, and send the acoustic imaging data and thermal imaging data during the autonomous imaging process of the acoustic imaging device to the backend server when it is determined to be the slave imaging device, so that the backend server forwards it to the determined master imaging device;

[0010] The acoustic imaging collaborative processing module is also used to acquire acoustic imaging data and thermal imaging data from other collaborative acoustic imaging devices during autonomous imaging according to the operating instructions of the acoustic imaging collaborative processing module. After performing spatiotemporal alignment processing on the acoustic imaging data and thermal imaging data from its own autonomous imaging process, it performs scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion respectively to obtain the final abnormal sound source sound field image of the target to be imaged, display the abnormal temperature, and send it to the imaging display device of the collaborative acoustic imaging device.

[0011] By adopting the above scheme, when the acoustic imaging device starts up, it sends coordination instructions and information to the backend server. The backend server then generates information to trigger the coordination mechanism based on the scene type, the number of online imaging devices, and topology requirements, thereby enabling the environmental data acquisition device to collect environmental data. The environmental data and performance index data are sent to the backend server to determine the master and slave imaging devices. The master imaging device can generate operating instructions, and the slave imaging device can transmit data to the master imaging device. Through spatiotemporal alignment processing and weighted fusion processing, the final sound field image of the abnormal sound source of the target to be imaged is obtained and the abnormal temperature is displayed, improving the accuracy and reliability of acoustic imaging. This method is suitable for monitoring abnormal sound sources and temperatures in different scenarios.

[0012] Preferably, the imaging display device includes: a display screen; a first data processing and analysis module for identifying abnormal sound signals of the target to be imaged using a first neural network model and generating a sound field image; and a second data processing and analysis module for obtaining abnormal temperatures using a second neural network model and annotating them on the sound field image. The first data processing and analysis module is further configured to match an acoustic imaging optimization algorithm to replace the first neural network model based on the scene type of the target to be imaged in order to identify abnormal sound signals of the target to be imaged. The acoustic imaging optimization algorithm has multiple types, and each type of acoustic imaging optimization algorithm is set with a matching scene type, including: a neural network model that considers environmental influences and a neural network model that increases reverberation effects.

[0013] The second data processing and analysis module is also used to match an environmental compensation model according to the scene type of the target to be imaged, and to perform real-time compensation for thermal imaging using the matched environmental compensation model to obtain a thermal image after environmental compensation; each environmental compensation model matched with the scene type adopts a deep learning algorithm and is generated by using environmental data collected under the actual scene type and the corresponding thermal imaging data as training samples.

[0014] The acoustic imaging collaborative processing module is further configured to acquire acoustic imaging data from other collaborative acoustic imaging devices during autonomous imaging, perform spatiotemporal alignment processing on the acoustic imaging data from its own autonomous imaging process, and then perform scene-type type-weighted fusion of acoustic imaging data. This fusion uses a weighted least squares method to fuse the acoustic imaging data from multiple acoustic imaging devices to determine the estimated location of abnormal sound sources. The weights are determined based on the signal-to-noise ratio and average time difference of arrival of the abnormal sound signals collected by multiple acoustic imaging devices under the corresponding scene type. The module is also configured to acquire thermal imaging data from other collaborative acoustic imaging devices during autonomous imaging, perform spatiotemporal alignment processing on the thermal imaging data from its own autonomous imaging process, and then perform scene-type-weighted fusion of thermal imaging data. This fusion uses a weighted least squares method to fuse the estimated location of abnormal temperature obtained by multiple acoustic imaging devices. The weights are determined based on the thermal imaging quality of multiple acoustic imaging devices under the corresponding scene type.

[0015] By adopting the above scheme, matching acoustic imaging optimization algorithms and environmental compensation models according to the type of the target scene to be imaged can improve the accuracy of abnormal sound signal identification and thermal imaging compensation effect; by using weighted least squares method to perform weighted fusion of acoustic and thermal imaging data, the location of abnormal sound sources and abnormal temperatures can be accurately determined, thereby improving the final imaging quality and accuracy.

[0016] Preferably, the acoustic imaging collaborative processing module is further configured to perform an acoustic-thermal consistency check when acquiring the final abnormal sound source sound field image of the target to be imaged and displaying the abnormal temperature; if the distance difference between the abnormal sound source location and the abnormal temperature location in the final acquired abnormal sound source sound field image is greater than the preset distance threshold for the corresponding scene type, the acoustic-thermal consistency check is deemed to have failed, and the collaborative acoustic imaging device self-calibration is initiated, so that the imaging display devices in each collaborative acoustic imaging device reacquire acoustic imaging data and thermal imaging data, and re-perform scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion, so as to reacquire the final abnormal sound source sound field image of the target to be imaged and display the abnormal temperature; and continue to determine whether the reacquired final abnormal sound source sound field image of the target to be imaged and display of the abnormal temperature passes the acoustic-thermal consistency check; if the acoustic-thermal consistency check still fails, the location of the abnormal sound source is marked as having a risk of instantaneous deflagration, and the location of the abnormal temperature is marked as having a potential smoldering fire source.

[0017] By adopting the above scheme, the array-type multi-channel acoustic imaging device can perform acoustic-thermal consistency testing. When the acoustic-thermal consistency testing fails, self-calibration is initiated, enabling each collaborative acoustic imaging display device to reacquire and fuse data to reacquire the final image and temperature information. It can also continuously test. If the test still fails, the location of abnormal sound source can be marked with an instantaneous deflagration risk, and the location of abnormal temperature can be marked with a potential smoldering fire source, thereby improving the accuracy and reliability of the imaging results and better identifying and warning of potential risks.

[0018] Preferably, the acoustic imaging collaborative processing module is further configured to, while acquiring the final sound field image of the abnormal sound source of the target to be imaged and displaying the abnormal temperature, obtain the confidence level of the abnormal sound source location; determine whether the confidence level of the obtained abnormal sound source location is greater than a preset confidence threshold for the corresponding scene type; and when the confidence level of the obtained abnormal sound source location is not greater than the preset confidence threshold for the corresponding scene type, generate a collaborative acoustic imaging device reconfiguration instruction to the backend server, so that when the backend server receives the collaborative acoustic imaging device reconfiguration instruction, it reselects collaborative acoustic imaging devices and determines the master and slave imaging devices in the collaborative acoustic imaging devices based on the environmental data and performance index data transmitted by each acoustic imaging device and the distance to the target to be imaged calculated based on the location of the imaging display device and the information of the target to be imaged, and ensures that the reselected collaborative acoustic imaging devices meet the minimum number of collaborative devices and topology requirements that match the scene type within a preset range from the target to be imaged.

[0019] By adopting the above scheme, when acquiring the sound field image of the abnormal sound source of the target to be imaged and displaying the abnormal temperature, the confidence level of the abnormal sound source location can be obtained. When the confidence level does not meet the preset threshold of the corresponding scene type, the collaborative acoustic imaging device can be reorganized, so that the backend server can reselect and determine the master and slave imaging devices, ensuring that the new collaborative acoustic imaging device meets the minimum number of collaborative devices and topology requirements, thereby improving the accuracy and reliability of imaging.

[0020] Preferably, the acoustic imaging collaborative processing module is further configured to transmit the final image of the abnormal sound source sound field of the target to be imaged and display the abnormal temperature to the background server.

[0021] The acoustic imaging collaboration request module is also used to mark the target information number when sending the target information to the backend server when the acoustic imaging device starts up, and simultaneously send the importance level and number of the target. This allows the backend server to statistically analyze the abnormal frequency of the historical abnormal sound source or abnormal temperature of each target when it determines that there are multiple targets to be imaged in the content sent by multiple sound field imaging devices. It also analyzes the danger level of the historical abnormal sound source or abnormal temperature obtained according to the danger level judgment rules of abnormal sound source or abnormal temperature. Based on the quantitative value of the abnormal frequency, the quantitative value of the danger level and the quantitative value of the importance of each target to be imaged in the statistical analysis results, it calculates a comprehensive quantitative value and generates the processing order of each target to be imaged in descending order of the calculated comprehensive quantitative value. Then, it executes the subsequent trigger collaboration mechanism information generation steps according to the processing order of each target to be imaged.

[0022] By adopting the above scheme, the acoustic imaging collaborative processing module transmits the sound field image and abnormal temperature of the abnormal sound source to the backend server, which facilitates unified management and data analysis. The acoustic imaging collaborative request module marks the information number of the target to be imaged, and synchronously sends the importance level and number, so that the backend server can perform statistical analysis on multiple targets to be imaged, calculate the comprehensive quantification value and generate the processing order, and then execute the trigger collaborative mechanism information generation steps in sequence. This helps to efficiently process different targets to be imaged and improve the efficiency and accuracy of acoustic imaging.

[0023] Preferably, the acoustic imaging collaborative processing module is further configured to determine the performance index of the current acoustic imaging device during the spatiotemporal alignment and weighted fusion processing of acoustic imaging data and thermal imaging data during its own autonomous imaging process. When the performance index of the current acoustic imaging device is determined to be lower than the preset performance index, the module will send the acoustic imaging data and thermal imaging data acquired during its own and other collaborative acoustic imaging devices' autonomous imaging processes to the backend server. This allows the backend server to perform spatiotemporal alignment processing based on the received acoustic imaging data and thermal imaging data from the main imaging device of the target to be imaged, and then perform scene-type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion to obtain the final abnormal sound source sound field image of the target to be imaged, display the abnormal temperature, and send it back to the imaging display device of all acoustic imaging devices.

[0024] By adopting the above scheme, when the performance index of the acoustic imaging device is lower than the preset performance index, the relevant imaging data can be sent to the back-end server for processing. The back-end server processes and fuses the data based on the main imaging device data benchmark, thereby obtaining an abnormal sound field image of the target to be imaged and displaying the abnormal temperature. The data is then sent back to all imaging display devices, which solves the problem of poor imaging effect that may be caused by insufficient device performance and ensures the accuracy and reliability of acoustic imaging and thermal imaging data processing.

[0025] Preferably, the acoustic imaging collaboration request module is used to generate, when the current acoustic imaging device reaches a preset number of master-slave imaging devices within a predetermined time period, a user-selectable instruction to directly stop sending imaging collaboration request commands, imaging display device location and / or target information to be imaged, or to send a target information importance upgrade instruction to the background server. This allows the background server to query the target information number sent by the corresponding acoustic imaging device according to the target information importance upgrade instruction, and upgrade the importance of the target information number of the queried target information.

[0026] By adopting the above scheme, excessive collaborative processing is prevented from failing to meet the current acoustic imaging device's ability to promptly meet user imaging needs. Users can choose to directly terminate collaborative processing or send an upgrade command for the importance of the target information to be imaged, which will enable the backend server to increase the importance of the corresponding target information. This helps the backend server to allocate resources more rationally and determine the processing order.

[0027] Secondly, this application discloses a method for locating and displaying abnormal sound sources using the aforementioned array-type multi-channel acoustic imaging device, comprising:

[0028] When the acoustic imaging device is started, the acoustic imaging coordination request module sends an imaging coordination request command, the location of the imaging display device, and / or the information of the target to be imaged to the backend server. This enables the backend server to determine, based on the scene type of the target to be imaged, whether the content sent by multiple acoustic field imaging devices meets the minimum number of coordination devices and topology requirements matching the scene type within a preset range of the target to be imaged, and to generate information to trigger the coordination mechanism. The backend server returns information on whether the coordination mechanism has been triggered, and when the information to trigger the coordination mechanism is received, an environmental data acquisition command is sent to the environmental data acquisition device.

[0029] Environmental data is collected using environmental data acquisition equipment after receiving an environmental data acquisition command;

[0030] The acoustic imaging collaboration request module sends the acquired environmental data and performance index data of the acoustic imaging device to the backend server. The backend server then determines the master and slave imaging devices in the collaborative acoustic imaging device based on the environmental data and performance index data transmitted by each acoustic imaging device, as well as the distance to the target to be imaged calculated based on the location of the imaging display device and the target information. The backend server receives the master and slave imaging device information returned by the backend server, determines whether it is the master imaging device of the target to be imaged, generates an acoustic imaging collaboration processing module running instruction when it is determined to be the master imaging device, and sends the acoustic imaging data and thermal imaging data during the autonomous imaging process of the acoustic imaging device to the backend server when it is determined to be the slave imaging device, so that the backend server forwards the data to the determined master imaging device.

[0031] The acoustic imaging co-processing module, according to its operating instructions, acquires acoustic imaging data and thermal imaging data from other co-acoustic imaging devices during their autonomous imaging processes. After performing spatiotemporal alignment processing on its own acoustic imaging data and thermal imaging data, scene-type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion are performed respectively to obtain the final abnormal sound source sound field image of the target to be imaged, display the abnormal temperature, and send it to the imaging display device of the co-acoustic imaging device.

[0032] By adopting the above scheme, the triggering coordination mechanism is determined and the master and slave imaging devices are reasonably identified. After spatiotemporal alignment and weighted fusion processing, the abnormal sound source sound field image and abnormal temperature of the target to be imaged are accurately acquired and displayed, thereby improving the accuracy and adaptability of acoustic imaging.

[0033] Thirdly, this application provides a computer-readable storage medium including a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to perform the method described above.

[0034] Fourthly, this application provides a computer device, the computer device including a memory, a processor and a program stored in the memory and executable thereon, the program being executed by the processor to implement the steps of the method described above.

[0035] In summary, this application has the following beneficial effects:

[0036] 1. By setting up an acoustic imaging collaborative request module to interact with the backend server, the system determines whether to trigger a collaborative mechanism based on the scene type of the target to be imaged. Based on the result of whether the collaborative mechanism is triggered, the system combines environmental data collected by the environmental data acquisition device to determine the master and slave imaging devices. This solves the problem of lack of device collaboration in existing technologies and improves the accuracy of abnormal sound source localization. The acoustic imaging collaborative processing module can perform spatiotemporal alignment and weighted fusion of acoustic imaging data and thermal imaging data. It can adaptively adjust according to the scene type, overcoming the shortcomings of existing technologies that cannot flexibly adjust algorithms and parameters according to scene changes, resulting in more ideal imaging effects.

[0037] 2. The acoustic imaging co-processing module can ensure the accuracy and consistency of acoustic imaging and acoustic thermal imaging data, promptly detect acoustic and thermal data deviations and automatically calibrate them. If multiple calibrations still fail to meet the requirements, high-risk locations will be clearly marked and prompted.

[0038] 3. By setting up an acoustic imaging collaborative processing module to acquire the sound field image of the abnormal sound source of the target to be imaged and display the abnormal temperature, the confidence level of the abnormal sound source location can be obtained. If the confidence level of the abnormal sound source location does not meet the requirements, the collaborative acoustic imaging device is reorganized in a timely manner to ensure that the reselected collaborative acoustic imaging device meets the scene requirements and improves the accuracy and reliability of acoustic imaging. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the array-type multi-channel acoustic imaging device described in a specific embodiment;

[0040] Figure 2 This is a flowchart illustrating the method for locating and displaying abnormal sound sources using the array-type multi-channel acoustic imaging device described in a specific embodiment. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0042] This application is based on a prior art array-type multi-channel acoustic imaging device and an abnormal sound source localization and display method. It further considers the collaboration of multiple acoustic imaging devices to achieve acoustic imaging and adjust as needed. This achieves the effects of accurately imaging the target in complex scenes, improving the accuracy of sound source localization, and obtaining temperature information. The following is a further detailed description of this application.

[0043] like Figure 1 As shown in the figure, this application discloses an array-type multi-channel acoustic imaging device, including an imaging display device 1, a main microphone array 2, a camera device 3, a thermal imaging device 4, an environmental data acquisition device 5, an acoustic imaging collaboration request module 6, and an acoustic imaging collaboration processing module 7; wherein, the main microphone array 2, the camera device 3, the thermal imaging device 4, and the environmental data acquisition device 5 are all installed on the imaging display device 1; the acoustic imaging collaboration request module 6 and the acoustic imaging collaboration processing module 7 establish communication with the imaging display device 1; in addition, each acoustic imaging device interacts with a background server, and the background server monitors all acoustic imaging devices in operation.

[0044] Specifically, the main microphone array 2 is used to collect the sound signal of the target to be imaged and transmit it to the imaging display device 1; taking the inspection of power transmission lines as an example, the target to be imaged is the power transmission line or power tower in the pre-assigned power transmission line area. The camera device 3 is used to collect the image of the target to be imaged and transmit it to the imaging display device 1; the thermal imaging device 4 is used to collect the thermal image of the target to be imaged and transmit it to the imaging display device 1; the imaging display device 1 includes: a display screen 11, a first data processing and analysis module 12 and a second data processing and analysis module 13 establishing a connection: the first data processing and analysis module 12 is used to identify abnormal sound signals of the target to be imaged using a first neural network model, and to preliminarily estimate the location of the abnormal sound source using a spatial positioning algorithm; using an acoustic imaging algorithm, it obtains the preliminarily estimated sound field distribution of the abnormal sound source, and combines it with the collected image of the target to be imaged. A sound field image is generated; the second data processing and analysis module 13 is used to obtain the rate of change of sound wave parameters per unit area in the sound field image; the obtained rate of change of sound wave parameters per unit area and the temperature of the corresponding unit area obtained from the thermal image are input into the second neural network model to obtain the sound velocity per unit area in the sound field image; the temperature of the unit area in the sound field image is calculated according to the relationship equation between sound velocity and gas temperature, and fused with the temperature of the corresponding unit area in the acquired thermal image to obtain the final temperature; the final temperature is compared with the preset temperature, an abnormal temperature prompt is generated and marked on the sound field image, and the sound field image marked with the abnormal temperature is displayed on the display screen.

[0045] To enable the acoustic imaging device to consider multi-device collaborative positioning and accurately image the target in complex scenes, meeting the multiple needs of improving the accuracy of abnormal sound source positioning and simultaneously acquiring temperature information, the acoustic imaging collaborative request module 6 provided in this application has data processing and instruction sending and receiving functions. When the acoustic imaging device is started, it sends an imaging collaborative request instruction, the location of the imaging display device, and / or the target information to the backend server. This allows the backend server to determine, based on the scene type of the target, whether the content received from multiple sound field imaging devices (i.e., the imaging collaborative request instruction, the location of the imaging display device, and / or the target information) meets the minimum number of collaborative devices and topology requirements matching the scene type within a preset range (different scene types of the target have preset ranges for matching the target) to generate trigger collaborative mechanism information (i.e., if the requirements are met, trigger collaborative mechanism information is generated). The module receives the trigger collaborative mechanism information returned by the backend server. Upon receiving trigger collaborative mechanism information, it sends an environmental data acquisition instruction to the environmental data acquisition device. If no trigger collaborative mechanism information is received, it continues to perform autonomous imaging using the imaging display device.

[0046] The acoustic imaging collaborative request module 6 sends instructions and information to the backend server in a specific data format, such as JSON, so that the backend server can accurately parse them. The acoustic imaging collaborative request module 6 allows preloading or receiving user-inputted target information to be imaged. Once target information to be imaged is preloaded or input, the corresponding target information is sent, including: target information encoding, target location information, and scene classification.

[0047] In this embodiment, power line monitoring is taken as an example. The scene types of the target to be imaged include: outdoor transmission towers, indoor power distribution rooms, underground cable tunnels, etc. Minimum number of devices and deployment requirements are set for each scene type. For example, the minimum number of collaborative devices for outdoor transmission towers is 3, and the deployment requirement is to meet the star topology (wide coverage, wind noise resistance); the minimum number of collaborative devices for indoor power distribution rooms is 4, and the deployment requirement is to meet the network topology (overcoming multipath effect); the minimum number of collaborative devices for underground cable tunnels is 4, and the deployment requirement is to meet the linear topology (long distance, narrow space setting).

[0048] The environmental data acquisition device 5 is used to collect environmental data after receiving an environmental data acquisition command, including collecting environmental data such as temperature and humidity, wind speed, and air pressure using temperature and humidity sensors, air pressure sensors, and wind speed sensors.

[0049] To better facilitate collaborative positioning of acoustic imaging devices, a master-slave topology is designed, consisting of one master imaging device and N slave imaging devices. The master imaging device is responsible for global data fusion and decision-making; the slave devices are only responsible for transmitting acoustic imaging data and thermal imaging data generated during the acquisition and autonomous imaging process. In this embodiment, the factors to be considered in determining the master and slave devices include the distance from the acoustic imaging device to the target to be imaged, the environmental stability of the acoustic imaging device, and the hardware performance of the acoustic imaging device itself. The acoustic imaging collaborative request module 6 is also used to send the acquired environmental data and the performance index data of the acoustic imaging device to the backend server, so that the backend server can determine the master and slave imaging devices in the collaborative acoustic imaging device based on the environmental data and performance index data transmitted by each acoustic imaging device and the distance to the target to be imaged calculated based on the location of the imaging display device and the information of the target to be imaged.

[0050] Specifically, the backend server determines the master and slave imaging devices using a weighted scoring method. Each factor is quantified, and a comprehensive score is calculated. The device with the highest comprehensive score is the master imaging device, and the remaining devices are selected as slave imaging devices based on their scores, satisfying a minimum number of devices minus one. The collaborative acoustic imaging device for determining the target information to be imaged is determined by factors such as: target distance (1 / 1+D), environmental stability (preset environmental stability quantification value corresponding to a specific environmental data range), and hardware performance (0.5*CPU utilization + 0.3*memory utilization + 0.2*battery fertility), with corresponding weights of 0.4, 0.3, and 0.3, respectively.

[0051] The acoustic imaging collaboration request module 6 is further configured to receive master-slave imaging device information returned by the backend server, determine whether it is the master imaging device of the target to be imaged, generate an acoustic imaging collaboration processing module running instruction when it is determined to be the master imaging device, and send acoustic imaging data and thermal imaging data during the autonomous imaging process of the acoustic imaging device to the backend server when it is determined to be the slave imaging device, so that the backend server forwards them to the determined master imaging device; wherein, the acoustic imaging data specifically includes the abnormal sound source location data initially estimated by the acoustic imaging device using the first data processing and analysis module 12, including sound field distribution information and related image data; the thermal imaging data specifically includes the final temperature data obtained by the acoustic imaging device using the second data processing and analysis module 13.

[0052] Considering that the acoustic imaging device is the main imaging device, responsible for global data fusion and decision-making, a corresponding acoustic imaging collaborative processing module is designed to complete multi-device data fusion and collaborative positioning. The acoustic imaging collaborative processing module 7 is also used to acquire acoustic imaging data and thermal imaging data from other collaborative acoustic imaging devices during their autonomous imaging process, according to the operating instructions of the acoustic imaging collaborative processing module. It performs spatiotemporal alignment processing on its own acoustic imaging data and thermal imaging data during autonomous imaging to ensure consistency in time and space between the data collected by different devices. Then, it performs scene-type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion, that is, it adaptively completes acoustic imaging data weighted fusion and thermal imaging data weighted fusion according to the current scene type. For example, different master-slave weight ratios are matched for weighted fusion of different scene types. Finally, it acquires the final abnormal sound source sound field image of the target to be imaged, displays the abnormal temperature, and sends it to the imaging display device of the collaborative acoustic imaging device (the master and slave devices determined by the backend server).

[0053] In addition, to further improve the accuracy of collaborative localization of abnormal sound sources, an acoustic imaging collaborative processing module 7 is designed. This module is used to obtain the confidence level of the abnormal sound source location while acquiring the final sound field image of the abnormal sound source of the target to be imaged and displaying the abnormal temperature. The confidence level of the abnormal sound source location is obtained by weighted calculation of the confidence level of the abnormal sound source location acquired by each sound field imaging device. The weights can be the same as the comprehensive score of the master and slave imaging devices mentioned above. The confidence level of the abnormal sound source location acquired by each sound field imaging device can be obtained by the first neural network model while acquiring the abnormal sound signal.

[0054] It is also used to determine whether the confidence level of obtaining the location of the abnormal sound source is greater than the preset confidence threshold (e.g., 90%) for the corresponding scene type. When the confidence level of obtaining the location of the abnormal sound source is not greater than the preset confidence threshold for the corresponding scene type, a collaborative acoustic imaging device reconfiguration instruction is generated and sent to the backend server. This allows the backend server to reselect collaborative acoustic imaging devices based on the environmental data and performance index data transmitted by each acoustic imaging device, as well as the distance to the target to be imaged calculated based on the location of the imaging display device and the information of the target to be imaged. The backend server also determines the master and slave imaging devices in the collaborative acoustic imaging devices and ensures that the reselected collaborative acoustic imaging devices meet the minimum number of collaborative devices and topology requirements that match the scene type within the preset range of the target to be imaged. Among them, reselecting a collaborative acoustic imaging device can be achieved by adjusting the weights in the comprehensive score calculation of each acoustic imaging device, recalculating the comprehensive score, and then reselecting the collaborative acoustic imaging device based on the calculated comprehensive score. This process continues until the confidence level of obtaining the location of the abnormal sound source for the reselected collaborative acoustic imaging device is greater than or equal to the preset confidence threshold for the corresponding scene type, or the number of times the collaborative acoustic imaging device is reselected reaches a preset threshold, at which point the reselection of the collaborative acoustic imaging device stops.

[0055] The implementation principle of this embodiment is as follows: the acoustic imaging collaborative request module interacts with the backend server to determine the collaborative mechanism and master-slave devices, and uses environmental data acquisition devices to collect environmental data to assist decision-making. The data collected by each acoustic imaging device is processed and fused by the acoustic imaging collaborative processing module, and finally presents accurate imaging results on the imaging display device. This enables flexible adjustments according to scene changes, ensures accurate imaging of abnormal sound sources, improves the accuracy of abnormal sound source localization, and can simultaneously acquire temperature information.

[0056] In a specific embodiment, to further improve the accuracy of acquiring the location of abnormal sound sources and abnormal temperatures, and to improve the final imaging quality and accuracy in different scenarios, the system further includes:

[0057] The first data processing and analysis module 12 is further used to match an acoustic imaging optimization algorithm to replace the first neural network model according to the scene type of the target to be imaged in order to identify abnormal sound signals of the target to be imaged; the acoustic imaging optimization algorithm has multiple types, and each type of acoustic imaging optimization algorithm is set with a matching scene type, including: a neural network model considering environmental impact matched with outdoor power transmission towers, and a neural network model increasing reverberation impact matched with indoor power distribution rooms or underground cable tunnels; wherein, the neural network model considering environmental impact includes environmental data and sound signals as inputs, and abnormal sound signals as outputs, and is generated by training through historical environmental data and historically labeled abnormal sound signals; the neural network model increasing reverberation impact has sound signals, environmental data, and reverberation parameters associated with the environmental data (obtained through specialized acoustic measurement equipment or a pre-set method) as inputs, and abnormal sound signals as outputs, and is generated by training through historical environmental data, reverberation parameters associated with the historical environmental data, and historically labeled abnormal sound signals.

[0058] The second data processing and analysis module 13 is also used to match an environmental compensation model according to the scene type of the target to be imaged, and to perform real-time compensation for thermal imaging using the matched environmental compensation model to obtain a thermal image after environmental compensation; each environmental compensation model matched with the scene type adopts a deep learning algorithm and is generated by using environmental data collected under the actual scene type and the corresponding thermal imaging data as training samples.

[0059] In addition to model optimization of the first data processing and analysis module 12 and the second data processing and analysis module 13, data fusion optimization is performed on the acoustic imaging collaborative processing module 7. Specifically, the acoustic imaging collaborative processing module 7 is also used to acquire acoustic imaging data from other collaborative acoustic imaging devices during autonomous imaging. After performing spatiotemporal alignment processing on the acoustic imaging data from its own autonomous imaging process, during the scene-type acoustic imaging data weighted fusion process, the estimated location of abnormal sound sources determined by fusing the acoustic imaging data corresponding to multiple acoustic imaging devices using the weighted least squares method is selected. The specific fusion weights are determined according to the phase... The signal-to-noise ratio (SNR) and average time-of-arrival (ATO) of abnormal sound signals collected by multiple acoustic imaging devices under different scene types are determined. Specifically, the SNR of abnormal sound signals collected by each acoustic imaging device and the ATO of abnormal sound signals arriving at different arrays are comprehensively calculated to obtain a comprehensive calculation value. A comprehensive calculation value range is set and matched for different scene types to determine the comprehensive calculation value range in which the comprehensive calculation value of each acoustic imaging device is located. A preset weight value matching the comprehensive calculation value range is then normalized to determine the ratio of the acoustic imaging data weight values ​​of multiple collaborative acoustic imaging devices.

[0060] The acoustic imaging collaborative processing module 7 is also used to acquire thermal imaging data from other collaborative acoustic imaging devices during autonomous imaging, perform spatiotemporal alignment processing on the thermal imaging data during its own autonomous imaging process, and then perform scene-type thermal imaging data weighted fusion. The weighted least squares method is used to fuse the estimated abnormal temperature location values ​​obtained by multiple acoustic imaging devices. The fusion weight can be determined according to the thermal imaging quality (such as sharpness, resolution and signal-to-noise ratio) acquired by multiple acoustic imaging devices under the corresponding scene type, and the weight ratio is the ratio of thermal imaging quality.

[0061] In a specific embodiment, during power transmission monitoring, the occurrence of abnormal sound sources is often accompanied by abnormal temperatures. Therefore, a sound-thermal consistency check can be performed on the identified locations of the abnormal sound sources and abnormal temperatures to further verify the accuracy of the current location. The system also includes:

[0062] The acoustic imaging collaborative processing module is further configured to perform an acoustic-thermal consistency check when acquiring the final abnormal sound source sound field image of the target to be imaged and displaying the abnormal temperature; if the distance difference between the abnormal sound source location and the abnormal temperature location in the final acquired abnormal sound source sound field image is greater than a preset distance threshold for the corresponding scene type, the acoustic-thermal consistency check is deemed to have failed, and the collaborative acoustic imaging device self-calibration is initiated, so that the imaging display devices in each collaborative acoustic imaging device reacquire acoustic imaging data and thermal imaging data, and re-perform scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion to reacquire acoustic imaging data and thermal imaging data. The system acquires the final sound field image of the abnormal sound source of the target to be imaged and displays the abnormal temperature. It then continues to determine whether the reacquired final sound field image of the abnormal sound source of the target to be imaged and displays the abnormal temperature passes the acoustic-thermal consistency test. If the acoustic-thermal consistency test still fails, there may be two situations: the first is that there may be an abnormal sound but the temperature is temporarily abnormal, that is, the location of the abnormal sound source is marked as having a risk of instantaneous deflagration; the second is that there is an abnormal temperature but no abnormal sound is displayed, that is, the location of the abnormal temperature is marked as having a potential smoldering fire source. The system then uses the marking to remind the inspection personnel to conduct timely inspections and maintenance.

[0063] In a specific embodiment, considering that multiple acoustic field imaging devices capable of forming a cooperative system can simultaneously upload multiple targets to be imaged, in order to orderly perform cooperative processing of multiple targets to be imaged, a comprehensive evaluation is conducted based on the importance level of different targets to be imaged, the frequency of possible anomalies of different targets to be imaged, and the danger level of anomalies, thereby determining the cooperative processing order of different targets to be imaged; the system further includes:

[0064] The acoustic imaging collaborative processing module 7 is also used to transmit the final abnormal sound source sound field image of the target to be imaged and the abnormal temperature to the background server, so that the background server stores the historical abnormal sound source sound field images of each target to be imaged with the abnormal temperature.

[0065] The acoustic imaging collaborative request module 6 is also used to mark the target information number (such as outdoor transmission tower W0012, indoor power distribution room R0007) when sending the target information to the backend server when the acoustic imaging device is started, and simultaneously send the importance level of the target (which can be divided into general, important, and very important according to whether the location of the target is in the central power supply area) and the number, so that when the backend server determines that there are multiple targets to be imaged in the content sent by multiple sound field imaging devices (such as outdoor transmission tower W0012, indoor power distribution room R0007 and underground cable tunnel F1180), it can statistically analyze the abnormal frequency of abnormal sound sources or abnormal temperatures of each target to be imaged in history, and the danger level of historical abnormal sound sources or abnormal temperatures obtained according to the abnormal sound source or abnormal temperature danger level judgment rules (such as: the number of abnormal sound sources is greater than a preset number threshold, which corresponds to the abnormal temperature danger level; the abnormal temperature is greater than a preset temperature threshold, which corresponds to the abnormal temperature danger level). The danger level is determined by the frequency of occurrence. The weighted comprehensive quantization value is calculated by summing the quantization values ​​of the abnormal frequency (0-1), the hazard level (0-1), and the importance (0-1) of each target to be imaged. For example, the abnormal frequency quantization value of outdoor transmission tower W0012 is 80%, corresponding to a quantization of 0.6; the most frequent occurrence of the historical abnormal sound source severity level on outdoor transmission tower W0012 corresponds to a quantization of 0.85; the importance level of outdoor transmission tower W0012 is moderate, corresponding to a quantization of 0.5. The final weighted comprehensive quantization value is 0.695, and the corresponding indoor power distribution room R00... The comprehensive quantization values ​​of 07 and underground cable tunnel F1180 are 0.725 and 0.875, respectively; and the processing order of each target to be imaged is generated according to the calculated comprehensive quantization value from largest to smallest (e.g., from first to last: underground cable tunnel F1180, indoor power distribution room R0007, and outdoor transmission tower W0012). The subsequent triggering coordination mechanism information generation steps are executed according to the processing order of each target to be imaged, such as: underground cable tunnel F1180, indoor power distribution room R0007, and outdoor transmission tower W0012 generating triggering coordination mechanism information in sequence.

[0066] Furthermore, considering that an acoustic imaging device may be located at the power monitoring center and will be continuously added to the collaborative positioning process, in order to avoid an acoustic imaging device being unable to process the imaging requirements of the corresponding assigned imaging target in a timely manner, the acoustic imaging collaborative request module 6 is designed to generate, for the user to choose, a direct stop sending imaging collaborative request instructions to the backend server, the imaging display device location and / or the imaging target information (i.e., indicating that the current acoustic imaging device is no longer participating in collaborative processing and is performing imaging processing on its own) or a target information importance upgrade instruction to the backend server, so that the backend server can query the target information number sent by the corresponding acoustic imaging device according to the target information importance upgrade instruction, such as: outdoor transmission tower W0012, and upgrade the importance of the target information number of the queried target information, such as: outdoor transmission tower W0012 is upgraded to important.

[0067] Correspondingly, the acoustic imaging collaboration request module 6 is also used to monitor that the user's selection to directly stop sending imaging collaboration request instructions, imaging display device location and / or target information to be imaged to the background server has reached a preset number of selections (such as 3 times), and when the acoustic imaging device is started again, it continues to send imaging collaboration request instructions, imaging display device location and / or target information to be imaged to the background server.

[0068] In a specific embodiment, considering that the main device needs to perform data fusion processing, and to address the issue of poor imaging results that may result from insufficient performance, thereby ensuring the accuracy and reliability of acoustic imaging and thermal imaging data processing and improving the overall imaging effect, the system further includes:

[0069] The acoustic imaging collaborative processing module 7 is further configured to determine the performance index of the current acoustic imaging device during the spatiotemporal alignment and weighted fusion processing of acoustic imaging data and thermal imaging data during its own autonomous imaging process. When the performance index of the current acoustic imaging device is determined to be lower than the preset performance index, the module will send the acoustic imaging data and thermal imaging data acquired during the autonomous imaging process of itself and other collaborative acoustic imaging devices to the background server. This allows the background server to perform spatiotemporal alignment processing based on the received acoustic imaging data and thermal imaging data during the imaging process of the main imaging device of the target to be imaged, and then perform scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion to obtain the final abnormal sound source sound field image of the target to be imaged, display the abnormal temperature, and send it back to the imaging display device of all acoustic imaging devices.

[0070] like Figure 2As shown, this application discloses a method for locating and displaying abnormal sound sources using the above-mentioned array-type multi-channel acoustic imaging device, the specific steps of which include:

[0071] S1. When the acoustic imaging device is started, the acoustic imaging coordination request module sends an imaging coordination request command, the location of the imaging display device and / or the target information to be imaged to the background server.

[0072] Specifically, the acoustic imaging collaboration request module sends an imaging collaboration request command, the location of the imaging display device, and / or the information of the target to be imaged to the backend server. This enables the backend server to determine, based on the scene type of the target to be imaged, whether the content sent by multiple acoustic field imaging devices meets the minimum number of collaborative devices and topology requirements that match the scene type within a preset range of the target to be imaged, so as to generate information to trigger the collaboration mechanism.

[0073] S2. Utilize the acoustic imaging collaboration request module to receive information from the backend server regarding whether the collaboration mechanism has been triggered, and upon receiving the information indicating that the collaboration mechanism has been triggered, send an environmental data acquisition command to the environmental data acquisition device.

[0074] S3. Collect environmental data using environmental data acquisition equipment after receiving the environmental data acquisition command.

[0075] S4. Use the acoustic imaging collaborative request module to send the acquired environmental data and the performance index data of the acoustic imaging device to the backend server.

[0076] Specifically, the acoustic imaging collaboration request module sends the acquired environmental data and the performance index data of the acoustic imaging device to the backend server. This allows the backend server to determine the master and slave imaging devices in the collaborative acoustic imaging device based on the environmental data and performance index data transmitted by each acoustic imaging device, as well as the distance to the target to be imaged calculated based on the location of the imaging display device and the target information.

[0077] S5. Utilize the acoustic imaging collaboration request module to receive master-slave imaging device information returned by the backend server, determine whether it is the master imaging device of the target to be imaged, generate an acoustic imaging collaboration processing module running instruction when it is determined to be the master imaging device, and send acoustic imaging data and thermal imaging data during the autonomous imaging process of the acoustic imaging device to the backend server when it is determined to be the slave imaging device, so that the backend server forwards the data to the determined master imaging device.

[0078] S6. Using the acoustic imaging collaborative processing module, according to the operation instructions of the acoustic imaging collaborative processing module, acquire acoustic imaging data and thermal imaging data during the autonomous imaging process of other collaborative acoustic imaging devices. After performing spatiotemporal alignment processing on the acoustic imaging data and thermal imaging data during its own autonomous imaging process, it performs scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion respectively to obtain the final abnormal sound source sound field image of the target to be imaged, display the abnormal temperature, and send it to the imaging display device of the collaborative acoustic imaging device.

[0079] This application also discloses a computer-readable storage medium.

[0080] Specifically, the computer-readable storage medium stores a computer program that can be loaded by a processor and executed as described in the abnormal sound source location display method. The computer-readable storage medium includes, for example, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0081] This application also discloses a computer device.

[0082] Specifically, the computer device includes a memory and a processor, and the memory stores a computer program that can be loaded by the processor and executed to display the abnormal sound source location method described above.

[0083] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. An array-type multi-channel acoustic imaging device, comprising: An imaging display device, a main microphone array, a camera device, and a thermal imaging device; characterized in that it further includes: an environmental data acquisition device, an acoustic imaging collaboration request module and an acoustic imaging collaboration processing module for establishing communication with the imaging display device; The acoustic imaging collaboration request module is used to send an imaging collaboration request instruction, the location of the imaging display device, and / or the information of the target to be imaged to the backend server when the acoustic imaging device is started. This allows the backend server to determine, based on the scene type of the target to be imaged, whether the content sent by multiple acoustic field imaging devices meets the minimum number of collaborative devices and topology requirements matching the scene type within a preset range of the target to be imaged, thereby generating information to trigger the collaboration mechanism. The module also receives information from the backend server indicating whether the collaboration mechanism has been triggered, and sends an environmental data acquisition instruction to the environmental data acquisition device when the information to trigger the collaboration mechanism is received. The environmental data acquisition device is used to acquire environmental data after receiving an environmental data acquisition command; The acoustic imaging collaboration request module is also used to send the acquired environmental data and performance index data of the acoustic imaging device to the backend server, so that the backend server can determine the master and slave imaging devices in the collaborative acoustic imaging device according to the environmental data and performance index data transmitted by each acoustic imaging device and the distance to the target to be imaged calculated according to the location of the imaging display device and the target information; receive the master and slave imaging device information returned by the backend server, determine whether it is the master imaging device of the target to be imaged, generate an acoustic imaging collaboration processing module running instruction when it is determined to be the master imaging device, and send the acoustic imaging data and thermal imaging data during the autonomous imaging process of the acoustic imaging device to the backend server when it is determined to be the slave imaging device, so that the backend server forwards it to the determined master imaging device; The acoustic imaging collaborative processing module is also used to acquire acoustic imaging data and thermal imaging data from other collaborative acoustic imaging devices during autonomous imaging according to the operating instructions of the acoustic imaging collaborative processing module. After performing spatiotemporal alignment processing on the acoustic imaging data and thermal imaging data from its own autonomous imaging process, it performs scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion respectively to obtain the final abnormal sound source sound field image of the target to be imaged, display the abnormal temperature, and send it to the imaging display device of the collaborative acoustic imaging device.

2. The array-type multi-channel acoustic imaging device according to claim 1, wherein the imaging display device comprises: The system comprises a display screen, a first data processing and analysis module for identifying abnormal sound signals of a target to be imaged using a first neural network model and generating a sound field image, and a second data processing and analysis module for obtaining abnormal temperatures using a second neural network model and annotating them on the sound field image. The first data processing and analysis module is further configured to match an acoustic imaging optimization algorithm to the scene type of the target to be imaged, replacing the first neural network model to identify abnormal sound signals of the target. The acoustic imaging optimization algorithm has multiple types, each with a matching scene type, including: a neural network model considering environmental influences and a neural network model incorporating reverberation effects. The second data processing and analysis module is also used to match an environmental compensation model according to the scene type of the target to be imaged, and to perform real-time compensation for thermal imaging using the matched environmental compensation model to obtain a thermal image after environmental compensation; each environmental compensation model matched with the scene type adopts a deep learning algorithm and is generated by using environmental data collected under the actual scene type and the corresponding thermal imaging data as training samples. The acoustic imaging collaborative processing module is also used to acquire acoustic imaging data during the autonomous imaging process of other collaborative acoustic imaging devices, perform spatiotemporal alignment processing on the acoustic imaging data during its own autonomous imaging process, and then perform scene-type acoustic imaging data weighted fusion. It uses a weighted least squares method to fuse the estimated location of abnormal sound sources determined by the acoustic imaging data corresponding to multiple acoustic imaging devices, with the weights determined based on the signal-to-noise ratio and average time difference of arrival of the abnormal sound signals collected by multiple acoustic imaging devices under the corresponding scene type. It is also used to acquire thermal imaging data during the autonomous imaging process of other collaborative acoustic imaging devices, perform spatiotemporal alignment processing on the thermal imaging data during its own autonomous imaging process, and then perform scene-type thermal imaging data weighted fusion. It uses a weighted least squares method to fuse the estimated location of abnormal temperature obtained by multiple acoustic imaging devices, with the weights determined based on the thermal imaging quality of multiple acoustic imaging devices under the corresponding scene type.

3. The array-type multi-channel acoustic imaging device according to claim 1, characterized in that, The acoustic imaging collaborative processing module is also used to perform an acoustic-thermal consistency check when acquiring the final abnormal sound source sound field image of the target to be imaged and displaying the abnormal temperature; if the distance difference between the abnormal sound source position and the abnormal temperature position in the final acquired abnormal sound source sound field image is greater than the preset distance threshold of the corresponding scene type, the acoustic-thermal consistency check is deemed to have failed, and the collaborative acoustic imaging device self-calibration is initiated so that the imaging display device in each collaborative acoustic imaging device reacquires acoustic imaging data and thermal imaging data, and re-performs scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion to reacquire the final abnormal sound source sound field image of the target to be imaged and display the abnormal temperature; It continues to evaluate the final re-acquired sound field image of the abnormal sound source of the target to be imaged and displays whether the abnormal temperature passes the acoustic-thermal consistency test; If the acoustic-thermal consistency test is still not passed, mark the location of the abnormal sound source as having a risk of instantaneous deflagration, and mark the location of the abnormal temperature as having a potential smoldering fire source.

4. The array-type multi-channel acoustic imaging device according to claim 1, characterized in that, The acoustic imaging collaborative processing module is further configured to, while acquiring the final sound field image of the abnormal sound source of the target to be imaged and displaying the abnormal temperature, obtain the confidence level of the abnormal sound source location; determine whether the confidence level of the obtained abnormal sound source location is greater than the preset confidence level threshold of the corresponding scene type; when the confidence level of the obtained abnormal sound source location is not greater than the preset confidence level threshold of the corresponding scene type, generate a collaborative acoustic imaging device reconfiguration instruction to the backend server, so that when the backend server receives the collaborative acoustic imaging device reconfiguration instruction, it reselects collaborative acoustic imaging devices and determines the master and slave imaging devices in the collaborative acoustic imaging devices based on the environmental data and performance index data transmitted by each acoustic imaging device and the distance to the target to be imaged calculated based on the location of the imaging display device and the information of the target to be imaged, and ensures that the reselected collaborative acoustic imaging devices meet the minimum number of collaborative devices and topology requirements that match the scene type within the preset range of the target to be imaged.

5. The array-type multi-channel acoustic imaging device according to claim 1, characterized in that, The acoustic imaging collaborative processing module is also used to transmit the final image of the abnormal sound source sound field of the target to be imaged and the abnormal temperature to the background server. The acoustic imaging collaboration request module is also used to mark the target information number when sending the target information to the backend server when the acoustic imaging device starts up, and simultaneously send the importance level and number of the target. This allows the backend server to statistically analyze the abnormal frequency of the historical abnormal sound source or abnormal temperature of each target when it determines that there are multiple targets to be imaged in the content sent by multiple sound field imaging devices. It also analyzes the danger level of the historical abnormal sound source or abnormal temperature obtained according to the danger level judgment rules of abnormal sound source or abnormal temperature. Based on the quantitative value of the abnormal frequency, the quantitative value of the danger level and the quantitative value of the importance of each target to be imaged in the statistical analysis results, it calculates a comprehensive quantitative value and generates the processing order of each target to be imaged in descending order of the calculated comprehensive quantitative value. Then, it executes the subsequent trigger collaboration mechanism information generation steps according to the processing order of each target to be imaged.

6. The array-type multi-channel acoustic imaging device according to claim 1, characterized in that, The acoustic imaging collaborative processing module is further configured to determine the performance index of the current acoustic imaging device during the spatiotemporal alignment and weighted fusion processing of acoustic imaging data and thermal imaging data during its own autonomous imaging process. When the performance index of the current acoustic imaging device is determined to be lower than the preset performance index, the module will send the acoustic imaging data and thermal imaging data acquired during the autonomous imaging process of itself and other collaborative acoustic imaging devices to the backend server. This allows the backend server to perform spatiotemporal alignment processing based on the received acoustic imaging data and thermal imaging data from the main imaging device of the target to be imaged, and then perform scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion to obtain the final abnormal sound source sound field image of the target to be imaged, display the abnormal temperature, and send it back to the imaging display device of all acoustic imaging devices.

7. The array-type multi-channel acoustic imaging device according to claim 5, characterized in that, The acoustic imaging collaboration request module is used to generate, when the current acoustic imaging device reaches a preset number of master-slave imaging devices within a predetermined time period, a user-selectable instruction to directly stop sending imaging collaboration request commands, imaging display device location and / or target information to be imaged, or to send a target information importance upgrade instruction to the background server. This allows the background server to query the target information number sent by the corresponding acoustic imaging device based on the target information importance upgrade instruction, and upgrade the importance of the target information number of the queried target information.

8. A method for locating and displaying abnormal sound sources using the array-type multi-channel acoustic imaging device according to any one of claims 1 to 7, characterized in that, include: When the acoustic imaging device is started, the acoustic imaging coordination request module sends an imaging coordination request command, the location of the imaging display device, and / or the information of the target to be imaged to the backend server. This enables the backend server to determine, based on the scene type of the target to be imaged, whether the content sent by multiple acoustic field imaging devices meets the minimum number of coordination devices and topology requirements matching the scene type within a preset range of the target to be imaged, and to generate information to trigger the coordination mechanism. The backend server returns information on whether the coordination mechanism has been triggered, and when the information to trigger the coordination mechanism is received, an environmental data acquisition command is sent to the environmental data acquisition device. Environmental data is collected using environmental data acquisition equipment after receiving an environmental data acquisition command; The acoustic imaging collaboration request module sends the acquired environmental data and performance index data of the acoustic imaging device to the backend server. The backend server then determines the master and slave imaging devices in the collaborative acoustic imaging device based on the environmental data and performance index data transmitted by each acoustic imaging device, as well as the distance to the target to be imaged calculated based on the location of the imaging display device and the target information. The backend server receives the master and slave imaging device information returned by the backend server, determines whether it is the master imaging device of the target to be imaged, generates an acoustic imaging collaboration processing module running instruction when it is determined to be the master imaging device, and sends the acoustic imaging data and thermal imaging data during the autonomous imaging process of the acoustic imaging device to the backend server when it is determined to be the slave imaging device, so that the backend server forwards the data to the determined master imaging device. The acoustic imaging co-processing module, according to its operating instructions, acquires acoustic imaging data and thermal imaging data from other co-acoustic imaging devices during their autonomous imaging processes. After performing spatiotemporal alignment processing on its own acoustic imaging data and thermal imaging data, scene-type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion are performed respectively to obtain the final abnormal sound source sound field image of the target to be imaged, display the abnormal temperature, and send it to the imaging display device of the co-acoustic imaging device.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the method as described in claim 8.

10. A computer device, characterized in that, The computer device includes a memory, a processor, and a program stored in and executable on the memory, the program being executed by the processor to implement the steps of the method as described in claim 8.

Images