Array type multi-channel acoustic imaging device and abnormal sound source positioning display method

Abstract

The invention discloses an array type multi-channel acoustic imaging device and an abnormal sound source positioning display method. The device comprises an acoustic imaging cooperation request module used for sending an imaging cooperation request instruction and related information of a to-be-imaged target to a background server, receiving returned information whether to trigger a cooperation mechanism and sending an environment data acquisition instruction; the environment data acquisition equipment is used for acquiring environment data; the acoustic imaging collaborative request module is also used for sending the acquired environment data and the performance index data of the acoustic imaging device to the background server; receiving returned master and slave imaging equipment information, and generating an acoustic imaging co-processing module operation instruction when the master imaging equipment is determined; and the acoustic imaging cooperative processing module is used for acquiring autonomous imaging data of the cooperative acoustic imaging device to carry out space-time alignment processing and data weighted fusion, and acquiring an abnormal sound source sound field image about the to-be-imaged target. According to the invention, the target to be imaged can be accurately imaged in a complex scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of acoustic imaging, in particular to an array type multi-channel acoustic imaging device and an abnormal sound source positioning display method. BACKGROUND

[0002] In the field of acoustic imaging technology, with the continuous development of industrial production, scientific research and other fields, the demand for accurate acquisition and analysis of sound information is increasing. Acoustic imaging technology can visualize sound, helping people intuitively understand the distribution and propagation of sound, which is of great significance in fault detection, noise source positioning, building acoustics evaluation and other aspects. It not only improves work efficiency, but also provides a powerful tool for solving complex acoustic problems, and promotes the technological progress in related fields.

[0003] In traditional acoustic imaging technology, in order to realize acoustic imaging, some conventional means are usually adopted. The common practice is to use a single microphone array to collect sound signals, and some existing technologies also use array type microphones to complete acoustic imaging, and combine with other imaging technologies such as thermal imaging and video imaging, further expanding its application range, so that sound, temperature and visual image information can be obtained at the same time. For example, the patent application with the application number CN202510088516.4 discloses an array type multi-channel acoustic imaging device, which realizes effective positioning of abnormal sound sources and display of abnormal sound sources and their temperature in complex environments, and improves the positioning accuracy and temperature measurement accuracy. However, the existing technology mainly relies on the independent work of a single device, and lacks cooperation with other devices. In addition, when facing different imaging scenes, fixed algorithms and parameters are often used, which cannot be flexibly adjusted according to the changes of the scene, and it is difficult to fully and accurately reflect the actual situation of the target scene, resulting in unsatisfactory imaging effect.

[0004] In summary, how to provide an acoustic imaging device that considers multi-device cooperative positioning and accurately images the imaging target in complex scenes, improves the positioning accuracy of abnormal sound sources, and simultaneously obtains temperature information for various demand applications is a technical problem that needs to be solved. SUMMARY

[0005] In order to provide an acoustic imaging device that considers multi-device cooperative positioning and accurately images the imaging target in complex scenes, improves the positioning accuracy of abnormal sound sources, and simultaneously obtains temperature information, the present application provides an array type multi-channel acoustic imaging device and an abnormal sound source positioning display method.

[0006] In a first aspect, the application provides an array type multi-channel acoustic imaging device, comprising: an imaging display device, a main microphone array, a camera device and a thermal imaging device; further comprising: an environmental data acquisition device, an acoustic imaging collaboration request module and an acoustic imaging collaboration processing module in communication with the imaging display device; The acoustic imaging collaboration request module is configured to, when the acoustic imaging device is started, send an imaging collaboration request instruction, an imaging display device position and / or a to-be-imaged target information to a background server, so that the background server determines whether the current received multiple sound field imaging device sending contents meet the distance to-be-imaged target preset range existing matching scene type The number of minimum collaboration devices and topology requirements to generate trigger collaboration mechanism information according to the scene type in which the to-be-imaged target is located; receive whether the trigger collaboration mechanism information returned by the background server, and send an environmental data acquisition instruction to the environmental data acquisition device when receiving the trigger collaboration mechanism information; The environmental data acquisition device is configured to collect environmental data after receiving the environmental data acquisition instruction; The acoustic imaging collaboration request module is further configured to send the acquired environmental data and performance index data of the acoustic imaging device to the background server, so that the background server determines the master and slave imaging devices in the collaborative acoustic imaging device according to the received environmental data, performance index data corresponding to each acoustic imaging device transmitted by the background server, and the distance to the to-be-imaged target calculated according to the imaging display device position and the to-be-imaged target information; receive the master and slave imaging device information returned by the background server, determine whether it is a master imaging device for the to-be-imaged target, generate an acoustic imaging collaboration processing module running instruction when it is determined to be a master imaging device, and send acoustic imaging data and thermal imaging data in the acoustic imaging device self-imaging process to the background server when it is determined to be a slave imaging device, so that the background server forwards to the determined master imaging device; The acoustic imaging collaboration processing module is further configured to, according to the acoustic imaging collaboration processing module running instruction, acquire acoustic imaging data and thermal imaging data in the self-imaging process of other collaborative acoustic imaging devices, and after time and space alignment processing according to the acoustic imaging data and thermal imaging data in the self-imaging process, respectively perform scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion, acquire the final abnormal sound source sound field image about the to-be-imaged target and display the abnormal temperature and send it to the imaging display device of the collaborative acoustic imaging device.

[0007] By adopting the above scheme, the acoustic imaging device sends a cooperative instruction and information to the background server when starting, so that the background server generates trigger cooperation mechanism information according to the scene type, the number of online imaging devices and the topology demand, and then makes the environment data acquisition equipment collect environment data; the environment data and performance index data are sent to the background server to determine the master-slave imaging equipment, the master imaging equipment can generate a running instruction, and the slave imaging equipment can transmit data to the master imaging equipment; through the space-time alignment processing and weighted fusion processing, the final abnormal sound source sound field image about the target to be imaged is obtained and the abnormal temperature is displayed, the accuracy and reliability of acoustic imaging are improved, and the abnormal sound source and temperature monitoring in different scenes are applicable.

[0008] Preferably, the imaging display device comprises: a display screen, a first data processing and analysis module for identifying an abnormal sound signal of a target to be imaged by using a first neural network model and generating a sound field image, and a second data processing and analysis module for obtaining an abnormal temperature by using a second neural network model and labeling the abnormal temperature on the sound field image, the first data processing and analysis module is further configured to replace the first neural network model with an acoustic imaging optimization algorithm matched according to a scene type in which the target to be imaged is located, to identify the abnormal sound signal of the target to be imaged; the acoustic imaging optimization algorithm has multiple types, and each type of acoustic imaging optimization algorithm is provided with a scene type matched thereto, including: a neural network model considering environmental impact and a neural network model increasing reverberation impact. The second data processing and analysis module is further configured to match an environmental compensation model according to the scene type in which the target to be imaged is located, to compensate the thermal imaging in real time by using the matched environmental compensation model, and to obtain a thermal image compensated by the environment; each environmental compensation model matched with the scene type is generated by using deep learning algorithm through environment data and corresponding thermal imaging data collected under an actual scene type as training samples; The acoustic imaging cooperative processing module is further configured to acquire acoustic imaging data in a self-imaging process of other cooperative acoustic imaging devices, to perform space-time alignment processing based on acoustic imaging data in a self-imaging process of the acoustic imaging device, and to perform scene type acoustic imaging data weighted fusion by using a weighted least square method to fuse abnormal sound source position estimation values determined by multiple acoustic imaging devices, wherein the weight is determined according to a signal-to-noise ratio and an average time difference of arrival of abnormal sound signals collected by the multiple acoustic imaging devices under the corresponding scene type; the acoustic imaging cooperative processing module is further configured to acquire thermal imaging data in a self-imaging process of other cooperative acoustic imaging devices, to perform space-time alignment processing based on thermal imaging data in a self-imaging process of the acoustic imaging device, and to perform scene type thermal imaging data weighted fusion by using a weighted least square method to fuse abnormal temperature position estimation values obtained by the multiple acoustic imaging devices, wherein the weight is determined according to thermal imaging quality of the multiple acoustic imaging devices under the corresponding scene type.

[0009] By adopting the above scheme, the abnormal sound signal recognition accuracy and the thermal imaging compensation effect can be improved according to the acoustic imaging optimization algorithm and the environment compensation model matched with the type of the target scene to be imaged; the abnormal sound source and the abnormal temperature position can be accurately determined by using the weighted least square method for acoustic and thermal imaging data weighted fusion, and the final imaging quality and accuracy can be improved.

[0010] Preferably, when the acoustic imaging cooperative processing module is further configured to acquire a final abnormal sound source sound field image of the target to be imaged and display an abnormal temperature, acoustic-thermal consistency verification is performed; if the distance difference between the abnormal sound source position and the abnormal temperature position in the final acquired abnormal sound source sound field image is greater than the preset distance threshold of the corresponding scene type, it is determined that the acoustic-thermal consistency verification fails, and the self-calibration of the cooperative acoustic imaging device is started, so that the imaging display devices in each cooperative acoustic imaging device reacquire acoustic imaging data and thermal imaging data, and re-perform scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion to reacquire a final abnormal sound source sound field image of the target to be imaged and display an abnormal temperature; and it is continuously determined whether the reacquired final abnormal sound source sound field image of the target to be imaged and the displayed abnormal temperature pass the acoustic-thermal consistency verification; if the acoustic-thermal consistency verification still fails, the abnormal sound source position is marked as having a risk of instantaneous explosion, and the abnormal temperature position is marked as having a potential smoldering fire source.

[0011] By adopting the above scheme, the array type multi-channel acoustic imaging device can perform acoustic-thermal consistency verification, start self-calibration when the acoustic-thermal consistency verification fails, make each cooperative acoustic imaging display device reacquire and fuse data to reacquire a final image and temperature information, and continuously perform verification; if the verification still fails, the abnormal sound source position can be marked as having a risk of instantaneous explosion, and the abnormal temperature position can be marked as having a potential smoldering fire source, so as to improve the accuracy and reliability of the imaging result, and better identify and warn potential risks.

[0012] Preferably, the acoustic imaging cooperative processing module is further configured to obtain a confidence level of the abnormal sound source position while obtaining the final abnormal sound source sound field image of the target to be imaged and displaying the abnormal temperature; determine whether the confidence level of the abnormal sound source position is greater than a preset confidence threshold of the corresponding scene type; and generate a cooperative acoustic imaging device reorganization instruction to the background server when the confidence level of the abnormal sound source position is not greater than the preset confidence threshold of the corresponding scene type, so that the background server reselects the cooperative acoustic imaging device and determines the master and slave imaging devices in the cooperative acoustic imaging device according to the environmental data, performance index data transmitted by each acoustic imaging device and the distance to the target to be imaged calculated according to the imaging display device position and the target to be imaged information when receiving the cooperative acoustic imaging device reorganization instruction, and ensures that the reselected cooperative acoustic imaging device meets the requirements of the minimum number of cooperative devices and topology within the preset range from the target to be imaged.

[0013] By using the above scheme, the confidence level of the abnormal sound source position can be obtained when the abnormal sound source sound field image of the target to be imaged is obtained and the abnormal temperature is displayed. When the confidence level does not meet the preset threshold of the corresponding scene type, the cooperative acoustic imaging device can be reorganized, the background server can reselect and determine the master and slave imaging devices, and the new cooperative acoustic imaging device can meet the requirements of the minimum number of cooperative devices and topology, thereby improving the accuracy and reliability of imaging.

[0014] Preferably, the acoustic imaging cooperative processing module is further configured to transmit the final abnormal sound source sound field image of the target to be imaged and the abnormal temperature to the background server. The acoustic imaging cooperative request module is further configured to send the target to be imaged information, mark the target to be imaged information number, and synchronously send the target to be imaged importance level and number when the acoustic imaging device is started, so that the background server can statistically analyze the abnormal frequency of each target to be imaged existing abnormal sound source or abnormal temperature and the historical abnormal sound source danger level or abnormal temperature danger level obtained according to the abnormal sound source or abnormal temperature danger level judgment rule when determining that there are multiple targets to be imaged in the content received by the multiple sound field imaging devices, and calculate the comprehensive quantitative value by weighting the abnormal frequency quantitative value, the danger level quantitative value and the target to be imaged importance quantitative value according to the statistical analysis result of each target to be imaged, and generate the processing order of each target to be imaged in descending order according to the calculated comprehensive quantitative value, and execute the subsequent cooperative mechanism information generation step according to the processing order of each target to be imaged.

[0015] By adopting the above scheme, the acoustic imaging cooperative processing module transmits the abnormal sound source sound field image and the abnormal temperature to the background server, so as to facilitate unified management and analysis of data; the acoustic imaging cooperative request module marks the to-be-imaged target information number, synchronously sends the importance level and the number, so that the background server can statistically analyze a plurality of to-be-imaged targets, calculate a comprehensive quantitative value, and generate a processing order, and then execute the information generation step of the trigger cooperative mechanism in order, which is helpful to efficiently process different to-be-imaged targets and improve the efficiency and accuracy of acoustic imaging.

[0016] Preferably, the acoustic imaging cooperative processing module is further configured to judge the performance index of the current acoustic imaging device during the spatio-temporal alignment processing and the weighted fusion processing of the acoustic imaging data and the thermal imaging data in the self-imaging process of the acoustic imaging device, and when the performance index of the current acoustic imaging device is lower than the preset performance index, send the acoustic imaging data and the thermal imaging data in the self-imaging process of the acoustic imaging device and other cooperative acoustic imaging devices to the background server, so that the background server performs spatio-temporal alignment processing based on the received acoustic imaging data and thermal imaging data in the imaging process of the to-be-imaged target, and then performs scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion respectively, to obtain the final abnormal sound source sound field image of the to-be-imaged target and display the abnormal temperature and return to the imaging display device of the acoustic imaging device.

[0017] By adopting the above scheme, when the performance index of the acoustic imaging device is lower than the preset performance index, the related imaging data can be sent to the background server for processing and fusion based on the data of the main imaging device, and then the abnormal sound source sound field image of the to-be-imaged target and the abnormal temperature are obtained and returned to all imaging display devices, so as to solve the problem of poor imaging effect caused by insufficient device performance, and ensure the accuracy and reliability of acoustic imaging and thermal imaging data processing.

[0018] Preferably, the acoustic imaging cooperative request module is configured to, when it is determined that the current acoustic imaging device is the main imaging device for a preset number of times within a preset period, generate a direct stop sending imaging cooperative request instruction to the background server, an imaging display device position and / or to-be-imaged target information, or send a to-be-imaged target information importance upgrade instruction to the background server, so that the background server queries the to-be-imaged target information number sent by the corresponding acoustic imaging device according to the to-be-imaged target information importance upgrade instruction, and upgrades the to-be-imaged target information of the queried to-be-imaged target information number.

[0019] By adopting the above scheme, excessive cooperative processing is prevented from causing the current acoustic imaging device to fail to meet the imaging requirements of the user in time, and the user can choose to directly terminate the cooperative processing or send an importance upgrade instruction of the to-be-imaged target information to make the background server upgrade the importance of the corresponding target information, which helps the background server to more reasonably allocate resources and determine the processing order.

[0020] In a second aspect, the application discloses an abnormal sound source positioning and display method applied to the above array type multi-channel acoustic imaging device, comprising: When the acoustic imaging device is started, an acoustic imaging cooperative request module is used to send an imaging cooperative request instruction, an imaging display device position, and / or to-be-imaged target information to a background server, so that the background server determines whether the current received content sent by multiple sound field imaging devices meets the requirement of the existence of a minimum number of cooperative devices matching the scene type within a preset range from the to-be-imaged target and a topology requirement to generate trigger cooperative mechanism information according to the scene type in which the to-be-imaged target is located; whether the trigger cooperative mechanism information is returned by the background server is received, and an environmental data acquisition instruction is sent to an environmental data acquisition device when the trigger cooperative mechanism information is received; The environmental data acquisition device is used to acquire environmental data after receiving the environmental data acquisition instruction; The acoustic imaging cooperative request module is used to send the acquired environmental data and performance index data of the acoustic imaging device to the background server, so that the background server determines master and slave imaging devices in the cooperative acoustic imaging device according to the received environmental data and performance index data corresponding to each acoustic imaging device transmitted by each acoustic imaging device and the distance to the to-be-imaged target calculated according to the imaging display device position and the to-be-imaged target information; master and slave imaging device information returned by the background server is received, whether the to-be-imaged target is a master imaging device is determined, an acoustic imaging cooperative processing module running instruction is generated when it is determined that the to-be-imaged target is a master imaging device, and acoustic imaging data and thermal imaging data in the self-imaging process of the acoustic imaging device are sent to the background server when it is determined that the to-be-imaged target is a slave imaging device, so that the background server forwards to the determined master imaging device; The acoustic imaging cooperative processing module is used to acquire acoustic imaging data and thermal imaging data in the self-imaging process of other cooperative acoustic imaging devices according to the acoustic imaging cooperative processing module running instruction, to perform scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion after time and space alignment processing of acoustic imaging data and thermal imaging data in the self-imaging process, to acquire an abnormal sound source sound field image about the to-be-imaged target and display abnormal temperature, and to send the abnormal sound source sound field image and the abnormal temperature to the imaging display device of the cooperative acoustic imaging device.

[0021] By adopting the above scheme, the cooperative mechanism is determined, the master and slave imaging devices are reasonably determined, the abnormal sound source sound field image and abnormal temperature of the to-be-imaged target are accurately acquired and displayed through space-time alignment and weighted fusion processing, and the accuracy and adaptability of acoustic imaging are improved.

[0022] In a third aspect, a computer readable storage medium is provided, which includes a stored computer program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to perform the method as described above when the computer program is running.

[0023] In a fourth aspect, a computer device is provided, which includes a memory, a processor, and a program stored in the memory and executable by the processor, and the program is executed by the processor to implement the steps of the method as described above.

[0024] In summary, the present application has the following beneficial effects: 1. The acoustic imaging cooperative request module is set to interact with the background server, whether the cooperative mechanism is triggered is determined according to the scene type where the to-be-imaged target is located, and the master and slave imaging devices are determined according to the result of whether the cooperative mechanism is triggered and combined with the environmental data collected by the environmental data collection device, thereby solving the problem of lack of device cooperation in the prior art and improving the accuracy of abnormal sound source positioning; the acoustic imaging cooperative processing module is set to perform space-time alignment processing and weighted fusion on acoustic imaging data and thermal imaging data, which can be adaptively adjusted according to the scene type, overcoming the defect that the prior art cannot flexibly adjust the algorithm and parameters according to the scene change, and making the imaging effect more ideal; 2. The acoustic imaging cooperative processing module can ensure the accuracy and consistency of acoustic imaging and acoustic-thermal imaging data, discover the acoustic-thermal data deviation problem in time and automatically calibrate, and when multiple calibrations still cannot meet the requirements, clearly mark and prompt the high-risk position; 3. When the acoustic imaging cooperative processing module acquires the abnormal sound source sound field image of the to-be-imaged target and displays the abnormal temperature, the position confidence of the abnormal sound source can be obtained, and when the position confidence of the abnormal sound source does not meet the requirements, the cooperative acoustic imaging device is reorganized in time to ensure that the newly selected cooperative acoustic imaging device meets the scene requirements, thereby improving the accuracy and reliability of acoustic imaging. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 FIG. 1 is a structural schematic diagram of an array type multi-channel acoustic imaging device described in the embodiments; Figure 2 FIG. 2 is a flowchart of an abnormal sound source positioning and display method using the array type multi-channel acoustic imaging device described in the embodiments. DETAILED DESCRIPTION

[0026] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not used to limit the present application.

[0027] The present application is mainly based on the prior art disclosed array multi-channel acoustic imaging device and abnormal sound source positioning display method, further considering the cooperation of multiple acoustic imaging devices, realizing acoustic imaging and on-demand adjustment scheme, achieving the effects of accurately imaging the imaging target in complex scenes, improving the accuracy of sound source positioning and obtaining temperature information. The present application will be further described in detail.

[0028] As shown in Figure 1 The present application discloses an array multi-channel acoustic imaging device, which comprises an imaging display device 1, a main microphone array 2, a camera device 3, a thermal imaging device 4, an environmental data acquisition device 5, an acoustic imaging cooperation request module 6 and an acoustic imaging cooperation processing module 7. The main microphone array 2, the camera device 3, the thermal imaging device 4 and the environmental data acquisition device 5 are all installed on the imaging display device 1. The acoustic imaging cooperation request module 6 and the acoustic imaging cooperation processing module 7 are in communication with the imaging display device 1. In addition, each acoustic imaging device has a background server interaction, and the background server monitors all acoustic imaging devices in running state.

[0029] Specifically, the main microphone array 2 is used to collect the sound signal of the target to be imaged and transmit it to the imaging display device 1; taking power transmission line inspection as an example, the target to be imaged is the power transmission line or power tower in the pre-allocated power transmission line area. The camera 3 is used to collect the image of the target to be imaged and transmit it to the imaging display device 1; the thermal imaging device 4 is used to collect the thermal image of the target to be imaged and transmit it to the imaging display device 1; the imaging display device 1 comprises a display screen 11, a first data processing and analysis module 12 and a second data processing and analysis module 13 connected: the first data processing and analysis module 12 is used to identify the abnormal sound signal of the target to be imaged by using the first neural network model, and preliminarily estimate the position of the abnormal sound source by using the spatial positioning algorithm; the sound field distribution of the preliminarily estimated abnormal sound source is obtained by using the acoustic imaging algorithm, and the sound field image is generated in combination with the collected image of the target to be imaged; the second data processing and analysis module 13 is used to obtain the change rate of the sound wave parameter per unit area in the sound field image; the obtained change rate of the sound wave parameter per unit area and the corresponding unit area temperature obtained according to the thermal image are input into the second neural network model to obtain the sound velocity per unit area in the sound field image; the temperature per unit area in the sound field image is calculated according to the relationship equation between the sound velocity and the gas temperature, and is fused with the corresponding unit area temperature in the collected thermal image to obtain the final temperature; the final temperature is compared with the preset temperature to generate an abnormal temperature prompt and mark it on the sound field image, and the display screen is used to display the sound field image marked with the abnormal temperature.

[0030] In order to enable the acoustic imaging device to consider multi-device cooperative positioning, accurately image the target to be imaged in a complex scene, and meet the multiple requirements of improving the positioning accuracy of the abnormal sound source and simultaneously obtaining temperature information, the acoustic imaging cooperative request module 6 provided by the present application has the functions of data processing and instruction sending and receiving, and is used to send an imaging cooperative request instruction, an imaging display device position and / or target to be imaged information to a background server when the acoustic imaging device starts, so that the background server determines whether the received content (i.e. imaging cooperative request instruction, imaging display device position and / or target to be imaged information) from multiple sound field imaging devices meets the existence of the minimum number of cooperative devices matching the scene type and the topological requirement within the preset range of the target to be imaged (the preset range of the target to be imaged matching different scene types is respectively preset) to generate trigger cooperative mechanism information (i.e. if it is met, the trigger cooperative mechanism information is generated); receive whether the trigger cooperative mechanism information returned by the background server, when the trigger cooperative mechanism information is received, send an environmental data acquisition instruction to the environmental data acquisition device; when the trigger cooperative mechanism information is not received, continue to use the imaging display device for autonomous imaging.

[0031] The instructions and information sent by the acoustic imaging cooperation request module 6 to the background server are encoded and sent in a specific data format, such as JSON format, so that the background server can accurately parse; the acoustic imaging cooperation request module 6 allows preloading or receiving user input of to-be-imaged target information, and once the to-be-imaged target information is preloaded or input, the to-be-imaged target information is sent, including to-be-imaged target information encoding, to-be-imaged target location information and scene classification.

[0032] In this embodiment, taking power line monitoring as an example, the scene types where the to-be-imaged target is located include outdoor power transmission towers, indoor power distribution rooms, underground cable tunnels, etc. The minimum number of devices and deployment requirements are set for each scene type, such as: the minimum number of cooperative devices for outdoor power transmission towers is 3, and the deployment requirement meets the star topology (wide coverage, wind noise resistance); the minimum number of cooperative devices for indoor power distribution rooms is 4, and the deployment requirement meets the network topology (overcoming multipath effect); the minimum number of cooperative devices for underground cable tunnels is 4, and the deployment requirement meets the linear topology (long distance, narrow space setting).

[0033] The environmental data acquisition device 5 is used to collect environmental data after receiving the environmental data acquisition instruction, including collecting temperature and humidity, wind speed, air pressure and other environmental data by using temperature and humidity sensors, air pressure sensors, wind speed sensors, etc.

[0034] In order to better perform cooperative positioning of the acoustic imaging device, a master-slave topology is designed, that is, one master imaging device and N slave imaging devices, the master imaging device is responsible for global data fusion and decision-making; the slave device is only responsible for transmitting acoustic imaging data and thermal imaging data generated in the process of self-imaging; in this embodiment, factors to be considered for determining the master-slave device include the distance from the acoustic imaging device to the to-be-imaged target, the environmental stability of the acoustic imaging device and the hardware performance of the acoustic imaging device itself, the acoustic imaging cooperation request module 6 is also used to send the acquired environmental data and performance index data of the acoustic imaging device to the background server, so that the background server determines the master-slave imaging device in the cooperative acoustic imaging device according to the received environmental data, performance index data of each acoustic imaging device and the distance from the to-be-imaged target calculated according to the imaging display device position and to-be-imaged target information; The method for determining the master and slave imaging devices by the specific background server is weighted scoring, each factor is quantified, and a comprehensive score is calculated by weighting. The imaging device with the highest comprehensive score is determined as the master imaging device. The remaining imaging devices are selected according to the scores, and the devices that meet the minimum device number-1 are determined as the slave imaging devices. The cooperative acoustic imaging device of the to-be-imaged target information is determined, such as target distance (1 / 1+D), environmental stability (a preset specific environmental data interval corresponding to a matching environmental stability quantization value), and hardware performance (0.5*CPU utilization rate+0.3*memory usage rate+0.2*battery growth rate). The corresponding weights are 0.4, 0.3, and 0.3, respectively.

[0035] The acoustic imaging cooperation request module 6 is also used to receive the master and slave imaging device information returned by the background server, determine whether the to-be-imaged target is the master imaging device, generate an acoustic imaging cooperation processing module running instruction when it is determined that the to-be-imaged target is the master imaging device, and send acoustic imaging data and thermal imaging data in the acoustic imaging device autonomous imaging process to the background server when it is determined that the to-be-imaged target is the slave imaging device, so that the background server forwards the acoustic imaging data and the thermal imaging data to the determined master imaging device. The acoustic imaging data specifically includes abnormal sound source position data estimated by the acoustic imaging device using the first data processing and analysis module 12, including sound field distribution information and related image data. The thermal imaging data specifically includes final temperature data obtained by the acoustic imaging device using the second data processing and analysis module 13.

[0036] Considering that the acoustic imaging device is the master imaging device responsible for global data fusion and decision-making, the acoustic imaging cooperation processing module is designed to complete multi-device data fusion and cooperative positioning. The acoustic imaging cooperation processing module 7 is also used to obtain acoustic imaging data and thermal imaging data in the autonomous imaging process of other cooperative acoustic imaging devices according to the acoustic imaging cooperation processing module running instruction, perform spatio-temporal alignment processing according to the acoustic imaging data and the thermal imaging data in the autonomous imaging process of the acoustic imaging device, ensure that the data collected by different devices are consistent in time and space, and then perform scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion, that is, acoustic imaging data weighted fusion and thermal imaging data weighted fusion are adaptively completed according to the current scene type, such as different scene types, different master-slave weight ratios are matched for weighted fusion, and finally the abnormal sound source sound field image of the to-be-imaged target is obtained and displayed. The abnormal temperature is sent to the imaging display device of the cooperative acoustic imaging device (the master and slave devices determined by the background server).

[0037] Further, in order to further improve the accuracy of the cooperative positioning of the abnormal sound source, the acoustic imaging cooperative processing module 7 is also used to obtain the confidence of the abnormal sound source position while obtaining the final abnormal sound field image of the target to be imaged and displaying the abnormal temperature; wherein the confidence of the abnormal sound source position is obtained by weighting the confidence of the abnormal sound source position obtained by each sound field imaging device, and the weight can be used as the comprehensive score of the master-slave imaging device determined above, and the confidence of the abnormal sound source position obtained by each sound field imaging device can be obtained by using the first neural network model while obtaining the abnormal sound signal; It is also used to judge whether the confidence of the abnormal sound source position is greater than the preset confidence threshold of the corresponding scene type (such as 90%), and when the confidence of the abnormal sound source position is not greater than the preset confidence threshold of the corresponding scene type, a cooperative acoustic imaging device reorganization instruction is generated to the background server, so that the background server reselects the cooperative acoustic imaging device and determines the master-slave imaging device in the cooperative acoustic imaging device according to the received environmental data, performance index data corresponding to each acoustic imaging device and the distance to the target to be imaged calculated according to the imaging display device position and the target to be imaged information when receiving the cooperative acoustic imaging device reorganization instruction, and ensures that the reselected cooperative acoustic imaging device meets the requirements of the minimum number of cooperative devices matching the scene type and topology within the preset range from the target to be imaged. Wherein, the reselected cooperative acoustic imaging device can adjust the weight in the comprehensive score calculation of each acoustic imaging device, recalculate the comprehensive score, and then select the cooperative acoustic imaging device according to the calculated comprehensive score, until the confidence of the abnormal sound source position is greater than or equal to the preset confidence threshold of the corresponding scene type or the number of times of reselecting the cooperative acoustic imaging device reaches the preset threshold, the step of reselecting the cooperative acoustic imaging device is stopped.

[0038] The implementation principle of the embodiment is: through the interaction between the acoustic imaging cooperative request module and the background server, the cooperative mechanism and the master-slave device are determined, and the environmental data acquisition device is used to collect environmental data to assist decision-making. The data collected by each acoustic imaging device is processed and fused by the acoustic imaging cooperative processing module, and finally an accurate imaging result is presented on the imaging display device, realizing flexible adjustment according to scene changes, ensuring accurate abnormal sound source imaging, improving the accuracy of abnormal sound source positioning and being able to obtain temperature information at the same time.

[0039] In a specific embodiment, in order to further improve the accuracy of the abnormal sound source and the abnormal temperature position, and improve the final imaging quality and accuracy in different scenes; the system further comprises: The first data processing and analysis module 12 is further configured to replace the first neural network model with an acoustic imaging optimization algorithm matched with a scene type in which the target to be imaged is located to identify abnormal sound signals of the target to be imaged. The acoustic imaging optimization algorithm has multiple types, and each type of acoustic imaging optimization algorithm is provided with a matched scene type. The acoustic imaging optimization algorithm includes a neural network model considering environmental impact matched with an outdoor power transmission tower, and a neural network model increasing reverberation impact matched with an indoor power distribution room or an underground cable tunnel. The neural network model considering environmental impact has input of environmental data and sound signals, and output of abnormal sound signals, and is generated by training historical environmental data and historical labeled abnormal sound signals. The neural network model increasing reverberation impact has input of sound signals, environmental data, and reverberation parameters associated with the environmental data (obtained by a special acoustic measurement device or a preset manner), and output of abnormal sound signals, and is generated by training historical environmental data, reverberation parameters associated with the historical environmental data, and historical labeled abnormal sound signals.

[0040] The second data processing and analysis module 13 is further configured to match an environmental compensation model according to a scene type in which the target to be imaged is located, to perform real-time compensation on thermal imaging by using the matched environmental compensation model, and to obtain a thermal image compensated by the environment. Each environmental compensation model matched with a scene type uses a deep learning algorithm, and is generated by using environmental data and corresponding thermal imaging data collected in an actual scene type as training samples.

[0041] In addition to model optimization of the first data processing and analysis module 12 and the second data processing and analysis module 13, the acoustic imaging cooperative processing module is optimized for data fusion, that is, the acoustic imaging cooperative processing module 7 is further configured to acquire acoustic imaging data in a self-imaging process of other cooperative acoustic imaging devices, to perform spatio-temporal alignment processing based on acoustic imaging data in a self-imaging process of the acoustic imaging cooperative processing module, and to select a weighted least square method to fuse abnormal sound source position estimation values determined by multiple acoustic imaging devices in a scene type acoustic imaging data weighting fusion process. Specifically, the fusion weight is determined according to a signal-to-noise ratio of an abnormal sound signal collected by each acoustic imaging device and an average time difference of arrival of the abnormal sound signal collected by each acoustic imaging device. That is, the signal-to-noise ratio of the abnormal sound signal collected by each acoustic imaging device and the average time difference of arrival of the abnormal sound signal collected by each acoustic imaging device are comprehensively calculated to obtain a comprehensive calculation value. For different scene types, a range of comprehensive calculation values matched with the scene types is set, the comprehensive calculation value range in which the comprehensive calculation value of each acoustic imaging device is located is determined, a preset weight value matched with the comprehensive calculation value range is determined, and the ratio of acoustic imaging data weight values of multiple cooperative acoustic imaging devices is normalized and determined.

[0042] The acoustic imaging cooperative processing module 7 is further configured to acquire thermal imaging data from other cooperative acoustic imaging devices in autonomous imaging processes, perform space-time alignment processing based on the thermal imaging data in the autonomous imaging process, and perform scene type thermal imaging data weighted fusion. The weighted least squares method is used to fuse abnormal temperature position estimation values obtained by multiple acoustic imaging devices. The fusion weight can be determined according to the thermal imaging quality (such as definition, resolution, and signal-to-noise ratio) collected by multiple acoustic imaging devices in a corresponding scene type. The ratio of the weights is the ratio of the thermal imaging quality.

[0043] In one specific embodiment, in the process of monitoring power transmission, once an abnormal sound source occurs, it is often accompanied by an abnormal temperature. Therefore, acoustic-thermal consistency verification can be performed on the determined position of the abnormal sound source and the position of the abnormal temperature, and the accuracy of the current positioning can be further verified. The system further comprises: When the acoustic imaging cooperative processing module acquires the final abnormal sound source sound field image of the target to be imaged and displays the abnormal temperature, acoustic-thermal consistency verification is performed. If the distance difference between the position of the abnormal sound source and the position of the abnormal temperature in the final acquired abnormal sound source sound field image is greater than the preset distance threshold of the corresponding scene type, it is determined that the acoustic-thermal consistency verification fails, and the cooperative acoustic imaging device is started for self-calibration. The imaging display device in each cooperative acoustic imaging device reacquires acoustic imaging data and thermal imaging data, and reperforms scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion to reacquire the final abnormal sound source sound field image of the target to be imaged and display the abnormal temperature. Whether the reacquired final abnormal sound source sound field image of the target to be imaged and the displayed abnormal temperature pass the acoustic-thermal consistency verification is continuously determined. If the acoustic-thermal consistency verification still fails, there can be two situations. The first situation is that there is an abnormal sound but a temporary temperature anomaly, that is, there is a risk of instantaneous explosion at the abnormal sound source position marker. The second situation is that there is an abnormal temperature but no abnormal sound is produced, that is, there is a potential smoldering fire source at the abnormal temperature position marker. Therefore, the inspection personnel are reminded to timely patrol and maintain through marker display.

[0044] In one specific embodiment, considering that at the same time, multiple acoustic field imaging devices can form a cooperation to simultaneously upload multiple targets to be imaged. In order to orderly perform cooperative processing of multiple targets to be imaged, the importance level of different targets to be imaged and the frequency and danger level of possible abnormalities of different targets to be imaged are comprehensively evaluated, and then the cooperative processing order of different targets to be imaged is determined. The system further comprises: The acoustic imaging cooperative processing module 7 is further configured to transmit the final abnormal sound source sound field image of the target to be imaged and the displayed abnormal temperature to the background server, so that the background server stores the historical abnormal sound source sound field image of the displayed abnormal temperature of each target to be imaged.

[0045] The acoustic imaging coordination request module 6 is further configured to, when the acoustic imaging device is started, send the to-be-imaged target information, the to-be-imaged target information number (such as outdoor power transmission tower W0012 and indoor power distribution room R0007), and the to-be-imaged target importance level (the importance level can be divided into general, important, and very important according to whether the to-be-imaged target is located in a central power supply area) to the background server at the same time, so that the background server determines that there are multiple to-be-imaged targets in the content received by the multiple acoustic field imaging devices (such as outdoor power transmission tower W0012, indoor power distribution room R0007, and underground cable tunnel F1180), analyzes the abnormal frequency of each to-be-imaged target existing abnormal sound source or abnormal temperature and the dangerous level of the historical abnormal sound source or the historical abnormal temperature (the frequency of occurrence is the dangerous level quantification standard) obtained according to the abnormal sound source or abnormal temperature dangerous level judgment rule (such as: the number of abnormal sound sources greater than the preset number threshold corresponds to the judgment of abnormal temperature dangerous level, and the abnormal temperature greater than the preset temperature threshold corresponds to the judgment of abnormal temperature dangerous level), and calculates the comprehensive quantification value (such as: the abnormal frequency quantification value of outdoor power transmission tower W0012 is 80%, which corresponds to 0.6; the most frequent occurrence of outdoor power transmission tower W0012 is the serious level of the historical abnormal sound source, which corresponds to 0.85, and the importance level of outdoor power transmission tower W0012 is general, which corresponds to 0.5, and the final weighted comprehensive quantification value is 0.695, and the corresponding comprehensive quantification values of indoor power distribution room R0007 and underground cable tunnel F1180 are 0.725 and 0.875, respectively) according to the abnormal frequency quantification value (0-1), the dangerous level quantification value (0-1), and the to-be-imaged target importance quantification value (0-1) in the statistical analysis result; and generates a to-be-imaged target processing order according to the calculated comprehensive quantification value from large to small (such as: from first to last: underground cable tunnel F1180, indoor power distribution room R0007, and outdoor power transmission tower W0012), and executes the subsequent trigger coordination mechanism information generation step according to the to-be-imaged target processing order, such as: underground cable tunnel F1180, indoor power distribution room R0007, and outdoor power transmission tower W0012 generate trigger coordination mechanism information in turn.

[0046] In addition, considering that an acoustic imaging device may be added to the cooperative positioning process at the power monitoring center position, to avoid that an acoustic imaging device cannot process the imaging requirements of the corresponding allocated target to be imaged in time, the acoustic imaging cooperative request module 6 is also used to generate a direct stop sending imaging cooperative request instructions, imaging display device positions and / or target to be imaged information to the background server (i.e., indicating that the current acoustic imaging device does not participate in cooperative processing, and performs imaging processing by itself) or sending target to be imaged information importance upgrade instructions to the background server, so that the background server queries the target to be imaged information number corresponding to the acoustic imaging device according to the target to be imaged information importance upgrade instruction, such as outdoor power transmission tower W0012, and upgrades the importance of the queried target to be imaged information number, such as upgrading the importance level of outdoor power transmission tower W0012 to important.

[0047] Correspondingly, the acoustic imaging cooperative request module 6 is also used to monitor that the user selects a direct stop sending imaging cooperative request instructions, imaging display device positions and / or target to be imaged information to the background server to reach a preset selection number (such as 3 times), and continue to send imaging cooperative request instructions, imaging display device positions and / or target to be imaged information to the background server at the next time the acoustic imaging device starts.

[0048] In one specific embodiment, considering that the master device needs to perform data fusion processing, to solve the problem of poor imaging effect caused by insufficient performance, to ensure the accuracy and reliability of acoustic imaging and thermal imaging data processing, and to improve the overall imaging effect, the system further comprises: The acoustic imaging cooperative processing module 7 is also used to judge the performance index of the current acoustic imaging device during the space-time alignment processing and weighted fusion processing of the acoustic imaging data and thermal imaging data in the self-imaging process of the acoustic imaging device and other cooperative acoustic imaging devices, and when it is judged that the performance index of the current acoustic imaging device is lower than the preset performance index, the acoustic imaging data and thermal imaging data in the self-imaging process of the acoustic imaging device and other cooperative acoustic imaging devices are sent to the background server, so that the background server performs space-time alignment processing based on the received acoustic imaging data and thermal imaging data of the target to be imaged in the imaging process of the master imaging device, and then performs scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion, respectively, to obtain the final abnormal sound source sound field image of the target to be imaged and display the abnormal temperature, and returns to the imaging display device of all acoustic imaging devices.

[0049] As shown in Figure 2 The present application discloses an abnormal sound source positioning display method using the above-mentioned array type multi-channel acoustic imaging device, and the specific steps include: S1, when the acoustic imaging device is started, an imaging collaboration request instruction, an imaging display device position and / or a to-be-imaged target information are sent to a background server by using an acoustic imaging collaboration request module.

[0050] Specifically, the imaging collaboration request instruction, the imaging display device position and / or the to-be-imaged target information are sent to the background server by using the acoustic imaging collaboration request module, so that the background server determines whether the current received content sent by a plurality of acoustic field imaging devices meets the requirement of the existence of a minimum number of collaborative devices matching the scene type and topology within a preset range from the to-be-imaged target to generate trigger collaboration mechanism information according to the scene type in which the to-be-imaged target is located.

[0051] S2, whether to trigger the collaboration mechanism information returned by the background server is received by using the acoustic imaging collaboration request module, and when the trigger collaboration mechanism information is received, an environmental data acquisition instruction is sent to an environmental data acquisition device.

[0052] S3, the environmental data is acquired by using the environmental data acquisition device after receiving the environmental data acquisition instruction.

[0053] S4, the acquired environmental data and performance index data of the acoustic imaging device are sent to the background server by using the acoustic imaging collaboration request module.

[0054] Specifically, the acquired environmental data and performance index data of the acoustic imaging device are sent to the background server by using the acoustic imaging collaboration request module, so that the background server determines the master and slave imaging devices in the collaborative acoustic imaging device according to the received environmental data and performance index data corresponding to each acoustic imaging device transmitted and the distance to the to-be-imaged target calculated according to the imaging display device position and the to-be-imaged target information.

[0055] S5, the master and slave imaging device information returned by the background server is received by using the acoustic imaging collaboration request module, whether it is a master imaging device of the to-be-imaged target is acquired, a acoustic imaging collaboration processing module running instruction is generated when it is determined to be a master imaging device, and acoustic imaging data and thermal imaging data in the acoustic imaging device self-imaging process are sent to the background server when it is determined to be a slave imaging device, so that the background server is forwarded to the determined master imaging device.

[0056] S6, the acoustic imaging cooperative processing module acquires acoustic imaging data and thermal imaging data in the autonomous imaging process of the other cooperative acoustic imaging device according to the acoustic imaging cooperative processing module running instruction, respectively performs scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion after space-time alignment processing according to the acoustic imaging data and the thermal imaging data in the autonomous imaging process of the acoustic imaging cooperative processing module, acquires an abnormal sound source sound field image about the target to be imaged and displays an abnormal temperature and sends the abnormal temperature to the imaging display device of the cooperative acoustic imaging device.

[0057] The embodiment of the present application further discloses a computer readable storage medium.

[0058] Specifically, the computer readable storage medium stores a computer program capable of being loaded and executed by the processor to implement the abnormal sound source positioning and display method, and the computer readable storage medium includes various storage medium capable of storing program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0059] The embodiment of the present application further discloses a computer device.

[0060] Specifically, the computer device includes a memory and a processor, and the memory stores a computer program capable of being loaded and executed by the processor to implement the abnormal sound source positioning and display method.

[0061] The above are preferred embodiments of the present application, and are not intended to limit the protection scope of the present application, and any feature disclosed in the specification (including the abstract and the drawings) can be replaced by other equivalent or similar purpose replacement features, unless specifically described. That is, each feature is only an example of a series of equivalent or similar features, unless specifically described.

Claims

1. An array-type multi-channel acoustic imaging device, comprising: Imaging display device, main microphone array, camera device and thermal imaging device; characterized by further comprising: environmental data acquisition device, acoustic imaging collaborative request module and acoustic imaging collaborative processing module for establishing communication with the imaging display device; The acoustic imaging collaborative request module is used to send an imaging collaborative request instruction, the location of the imaging display device and / or information of the target to be imaged to the background server when the acoustic imaging device is started, so that the background server can determine whether the content currently received from multiple sound field imaging devices meets the minimum number of collaborative devices and topology requirements matching the scene type within a preset range from the target to be imaged, so as to generate triggering collaborative mechanism information; receive whether the collaborative mechanism is triggered information returned by the background server, and send an environmental data collection instruction to the environmental data collection device when the triggering collaborative mechanism information is received; The environmental data collection device is used to collect environmental data after receiving an environmental data collection instruction; The acoustic imaging collaborative request module is further used to send the acquired environmental data and the performance indicator data of the acoustic imaging device to the backend server, so that the backend server determines the master and slave imaging devices in the collaborative acoustic imaging device according to the received environmental data and performance indicator data transmitted by each acoustic imaging device and the distance to the target to be imaged calculated according to the position of the imaging display device and the information of the target to be imaged; receives the master and slave imaging device information returned by the backend server, obtains whether it is the master imaging device of the target to be imaged, generates an acoustic imaging collaborative processing module operation instruction when it is determined to be the master imaging device, and sends the acoustic imaging data and thermal imaging data of the acoustic imaging device during the autonomous imaging process to the backend server when it is determined to be the slave imaging device, so that the backend server forwards them to the determined master imaging device; The acoustic imaging collaborative processing module is also used to obtain acoustic imaging data and thermal imaging data during the autonomous imaging process of other collaborative acoustic imaging devices according to the operation instructions of the acoustic imaging collaborative processing module, and after performing spatiotemporal alignment processing on the acoustic imaging data and thermal imaging data during its own autonomous imaging process, perform scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion respectively to obtain the final abnormal sound source sound field image of the target to be imaged, display the abnormal temperature, and send it to the imaging display device of the collaborative acoustic imaging device.

2. The array-type multi-channel acoustic imaging apparatus according to claim 1, wherein the imaging display device comprises: A display screen, a first data processing and analysis module for using a first neural network model to identify abnormal sound signals of a target to be imaged and generate a sound field image, and a second data processing and analysis module for using a second neural network model to obtain abnormal temperatures and mark them on the sound field image, characterized in that the first data processing and analysis module is further used to replace the first neural network model with an acoustic imaging optimization algorithm according to the scene type in which the target to be imaged is located to identify abnormal sound signals of the target to be imaged; the acoustic imaging optimization algorithm has multiple types, and each type of acoustic imaging optimization algorithm is provided with a scene type that matches it, including: a neural network model that considers environmental influences and a neural network model that adds reverberation influences; The second data processing and analysis module is further configured to match an environmental compensation model according to the scene type in which the target to be imaged is located, and to perform real-time compensation on the thermal image using the matched environmental compensation model to obtain an environmentally compensated thermal image; each environmental compensation model matched to the scene type is generated using a deep learning algorithm, using environmental data collected under the actual scene type and the corresponding thermal imaging data as training samples; The acoustic imaging collaborative processing module is also used to obtain acoustic imaging data during the autonomous imaging process of other collaborative acoustic imaging devices, and after performing spatiotemporal alignment processing on the acoustic imaging data during its own autonomous imaging process, perform weighted fusion of scene type acoustic imaging data by using the weighted least squares method to fuse abnormal sound source position estimates determined by the corresponding acoustic imaging data of multiple acoustic imaging devices, and the weights are determined according to the signal-to-noise ratio and average arrival time difference of abnormal sound signals collected by multiple acoustic imaging devices under the corresponding scene type; it is also used to obtain thermal imaging data during the autonomous imaging process of other collaborative acoustic imaging devices, and after performing spatiotemporal alignment processing on the thermal imaging data during its own autonomous imaging process, perform weighted fusion of scene type thermal imaging data by using the weighted least squares method to fuse abnormal temperature position estimates obtained by multiple acoustic imaging devices, and the weights are determined according to the thermal imaging quality of multiple acoustic imaging devices under the corresponding scene type.

3. The array-type multi-channel acoustic imaging device according to claim 1, characterized in that: The acoustic imaging collaborative processing module is further configured to perform an acoustic-thermal consistency test when obtaining a final abnormal sound source sound field image of the target to be imaged and displaying an abnormal temperature; if the distance difference between the abnormal sound source position and the abnormal temperature position in the final abnormal sound source sound field image is greater than a preset distance threshold for the corresponding scene type, the acoustic-thermal consistency test is deemed to have failed, and the collaborative acoustic imaging device self-calibration is initiated, so that the imaging display device in each collaborative acoustic imaging device reacquires the acoustic imaging data and the thermal imaging data, and re-performs the scene type adaptive acoustic imaging data weighted fusion and the thermal imaging data weighted fusion, so as to re-acquire the final abnormal sound source sound field image of the target to be imaged and display the abnormal temperature; and continuing to judge whether the reacquired final abnormal sound source sound field image of the target to be imaged and the abnormal temperature have passed the acoustic-thermal consistency test; If the acoustic and thermal consistency test is still not passed, there is a risk of instantaneous explosion at the abnormal sound source position mark, and there is a potential smoldering fire source at the abnormal temperature position mark.

4. The array-type multi-channel acoustic imaging device according to claim 1, characterized in that: The acoustic imaging collaborative processing module is also used to obtain the confidence of the abnormal sound source position while obtaining the final abnormal sound source sound field image of the target to be imaged and displaying the abnormal temperature; determine whether the confidence of the abnormal sound source position is greater than the confidence threshold preset for the corresponding scene type, and when the confidence of the abnormal sound source position is not greater than the confidence threshold preset for the corresponding scene type, generate a collaborative acoustic imaging device reorganization instruction to the background server, so that when the background server receives the collaborative acoustic imaging device reorganization instruction, it reselects the collaborative acoustic imaging device and determines the master and slave imaging devices in the collaborative acoustic imaging device based on the environmental data and performance indicator data transmitted by each acoustic imaging device and the distance to the target to be imaged calculated based on the imaging display device position and the target to be imaged information, and ensures that the reselected collaborative acoustic imaging device meets the minimum number of collaborative devices and topology requirements that match the scene type within the preset range from the target to be imaged.

5. The array-type multi-channel acoustic imaging device according to claim 1, characterized in that: The acoustic imaging collaborative processing module is further used to obtain the final abnormal sound source sound field image of the target to be imaged and display the abnormal temperature and transmit it to the background server; The acoustic imaging collaborative request module is also used to mark the information number of the target to be imaged when sending the information of the target to be imaged to the background server when the acoustic imaging device is started, and synchronously send the importance level and number of the target to be imaged, so that when the background server determines that there are multiple targets to be imaged in the content currently received from multiple sound field imaging devices, it statistically analyzes the abnormal frequency of abnormal sound sources or abnormal temperatures in the history of each target to be imaged and the danger level of the historical abnormal sound source or abnormal temperature obtained according to the abnormal sound source or abnormal temperature danger level judgment rules, and calculates a comprehensive quantitative value based on the quantitative value of the abnormal frequency, the quantitative value of the danger level and the quantitative value of the importance of each target to be imaged in the statistical analysis results, generates a processing order for each target to be imaged from large to small according to the calculated comprehensive quantitative value, and executes the subsequent triggering collaborative mechanism information generation step according to the processing order of each target to be imaged.

6. The array-type multi-channel acoustic imaging device according to claim 1, characterized in that: The acoustic imaging collaborative processing module is also used to judge the performance indicators of the current acoustic imaging device during the process of performing spatiotemporal alignment processing and weighted fusion processing based on the acoustic imaging data and thermal imaging data in its own autonomous imaging process. When it is judged that the performance indicators of the current acoustic imaging device are lower than the preset performance indicators, the acoustic imaging data and thermal imaging data obtained during the autonomous imaging process of itself and other collaborative acoustic imaging devices will be sent to the background server, so that the background server performs spatiotemporal alignment processing based on the acoustic imaging data and thermal imaging data received during the imaging process of the main imaging device of the target to be imaged, and then performs scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion respectively to obtain the final abnormal sound source sound field image of the target to be imaged and display the abnormal temperature and transmit it back to the imaging display devices of all acoustic imaging devices.

7. The array-type multi-channel acoustic imaging device according to claim 5, characterized in that: The acoustic imaging collaborative request module is used to generate a direct stop instruction for the user to send an imaging collaborative request instruction, an imaging display device position and / or information of a target to be imaged to a background server, or to send an importance upgrade instruction for the target to be imaged information to the background server when it is determined that the current acoustic imaging device has been the master or slave imaging device for a preset number of times within a preset time period, so that the background server can query the target to be imaged information number sent by the corresponding acoustic imaging device according to the importance upgrade instruction for the target to be imaged information, and upgrade the importance of the target to be imaged information of the queried target to be imaged information number.

8. A method for locating and displaying abnormal sound sources using the array multi-channel acoustic imaging device according to any one of claims 1 to 7, characterized in that: include: When the acoustic imaging device is started, the acoustic imaging collaborative request module is used to send an imaging collaborative request instruction, the location of the imaging display device and / or information of the target to be imaged to the background server, so that the background server can determine whether the content currently received from multiple sound field imaging devices meets the minimum number of collaborative devices and topology requirements that match the scene type within a preset range from the target to be imaged, based on the scene type of the target to be imaged, so as to generate triggering collaborative mechanism information; receive the information on whether to trigger the collaborative mechanism returned by the background server, and send an environmental data collection instruction to the environmental data collection device when receiving the triggering collaborative mechanism information; Using environmental data collection equipment to collect environmental data after receiving an environmental data collection instruction; The acoustic imaging collaborative request module is used to send the acquired environmental data and the performance indicator data of the acoustic imaging device to the backend server, so that the backend server determines the master and slave imaging devices in the collaborative acoustic imaging device according to the environmental data and performance indicator data transmitted by each acoustic imaging device and the distance to the target to be imaged calculated according to the position of the imaging display device and the information of the target to be imaged; the master and slave imaging device information returned by the backend server is received, and whether it is the master imaging device of the target to be imaged is obtained. When it is determined to be the master imaging device, an acoustic imaging collaborative processing module operation instruction is generated; when it is determined to be the slave imaging device, the acoustic imaging data and thermal imaging data during the autonomous imaging process of the acoustic imaging device are sent to the backend server, so that the backend server forwards them to the determined master imaging device; The acoustic imaging collaborative processing module is used to obtain the acoustic imaging data and thermal imaging data of other collaborative acoustic imaging devices during the autonomous imaging process according to the acoustic imaging collaborative processing module operation instructions. After performing spatiotemporal alignment processing on the acoustic imaging data and thermal imaging data during its own autonomous imaging process, scene type adaptive acoustic imaging data weighted fusion and thermal imaging data weighted fusion are performed respectively to obtain the final abnormal sound source sound field image of the target to be imaged, display the abnormal temperature, and send it to the imaging display device of the collaborative acoustic imaging device.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to claim 8.

10. A computer device, characterized in that: The computer device includes a memory, a processor, and a program stored and executable on the memory, and the steps of the method according to claim 8 are implemented when the program is executed by the processor.

Images