Air conditioner production line abnormal sound detection method and system

Abstract

The application provides an air conditioner production line abnormal sound detection method and system, comprising: acquiring sound signals in real time through a microphone spherical array; processing the sound signals based on an HOA-SHT domain analysis framework to generate a panorama sound image; acquiring an air conditioner image through a panorama camera; locking an air conditioner area according to the air conditioner image; performing multi-modal positioning on the air conditioner area through the panorama sound image to obtain a target area; extracting sound signals from the target area through a multi-scale mel spectrum, and inputting the extracted sound signals into a beamformer of a neural network to obtain noise signals; detecting and identifying the noise signals based on a convolutional neural network-based anomaly classifier to obtain air conditioner abnormal sound classification results; efficiently and accurately detecting air conditioner hanging machine abnormal sounds, effectively overcoming the limitations of traditional manual detection methods, significantly improving production efficiency, reducing the false detection rate and the missed detection rate, and providing strong quality guarantee for the production line.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of air conditioner detection, in particular to an abnormal sound detection method and system for air conditioner production line. BACKGROUND

[0002] Air conditioner hanging machine as an important component of air conditioning system, plays a key role in the process of refrigeration, heating and air circulation. Among them, the wind wheel blade as one of the core components of air conditioner hanging machine, its performance directly affects the running efficiency of air conditioner and user experience.

[0003] In the production process of air conditioner hanging machine, the manufacturing and assembly of wind wheel blade are crucial. However, in actual production, due to the influence of raw materials, production environment, process flow and other factors, foreign matters such as dust, particulate matter, oil stains or other production residues may adhere to the surface of wind wheel blade. The adhesion of these foreign matters on the surface of wind wheel blade may cause abnormal vibration of the blade during operation, thereby causing noise problem.

[0004] When the air conditioner runs, the rotation speed of the wind wheel blade is high, especially in the refrigeration or heating mode, the high-speed operation of the blade is easy to bring the adhered foreign matters into the airflow, causing friction or collision between the blade and the foreign matters, producing additional noise. This noise may be covered by environmental noise during the day, but at night, when the environmental noise is low, the weak noise may also become abnormally obvious, seriously affecting the sleep quality and life experience of users.

[0005] The existing method mainly relies on manual sound screening, and workers can quickly screen air conditioner hanging machines in the production line through auditory perception of abnormal sound, so as to discover abnormalities in time. This method has the characteristics of low cost, simple operation and rapid judgment. However, this method has significant limitations: first, the judgment mainly depends on the auditory perception and experience of workers, lacks unified quantitative standard, and the result is easy to vary from person to person, which is subjective; second, it is difficult to form objective data support for accurately measuring and recording the type, intensity or position of abnormal sound; in addition, workers are easy to be tired in long-time and high-intensity work, which leads to the decrease of judgment accuracy and high false rejection rate; finally, manual sound screening needs workers to check one by one, which is low in efficiency and difficult to meet the high rhythm demand of large-scale production. SUMMARY

[0006] Therefore, the purpose of the present application is to provide an abnormal sound detection method and system for air conditioner production line, to realize efficient and accurate detection of abnormal sound of air conditioner hanging machine, effectively overcome the limitations of traditional manual detection method, significantly improve production efficiency, reduce false rejection rate and omission rate, and provide strong guarantee for production line quality.

[0007] In a first aspect, the embodiments of the present application provide an abnormal sound detection method for air conditioner production line, which comprises:

[0008] acquire sound signals in real time through a microphone spherical array;

[0009] process the sound signals based on a HOA-SHT domain analysis framework to generate a panorama sound image;

[0010] acquire an air conditioner image through a panoramic camera;

[0011] lock an air conditioner region according to the air conditioner image;

[0012] perform multi-modal positioning on the air conditioner region through the panorama sound image to obtain a target region;

[0013] extract sound signals from the target region through multi-scale mel spectra, and input the extracted sound signals into a beamformer of a neural network to obtain noise signals;

[0014] detect and identify the noise signals based on a convolutional neural network-based anomaly classifier to obtain an air conditioner abnormal sound classification result.

[0015] Further, processing the sound signals based on the HOA-SHT domain analysis framework to generate a panorama sound image includes:

[0016] preprocessing the sound signals through a Hanning window short-time Fourier transform to obtain preprocessed sound signals;

[0017] performing spherical harmonic domain transformation on the preprocessed sound signals to obtain a 7th-order surround sound field decomposition;

[0018] performing spherical harmonic coefficient energy compensation on the 7th-order surround sound field decomposition to obtain spherical harmonic domain features;

[0019] obtaining a sound field through a beamformer based on the spherical harmonic domain features;

[0020] obtaining various spatial orientation features through a panoramic beam space based on the sound field;

[0021] superimposing the various spatial orientation features and the air conditioner image to obtain the panorama sound image.

[0022] Further, locking the air conditioner region according to the air conditioner image includes:

[0023] inputting the air conditioner image into a target detection model to obtain a spatial position of the air conditioner;

[0024] locking the air conditioner region according to the spatial position of the air conditioner.

[0025] Further, the method further includes:

[0026] Real-time monitoring the running state of the production line through the panoramic camera;

[0027] When the running state of the production line is a shutdown state, controlling audio and video collection to be paused.

[0028] Further, the method further comprises:

[0029] The air conditioner abnormal sound classification result is prompted through voice broadcast.

[0030] Further, the air conditioner abnormal sound classification result includes foreign matter adhesion, blade burr and sheet metal vibration.

[0031] In a second aspect, the embodiments of the present application provide an abnormal sound detection system for an air conditioner production line, which comprises a computing server and a spherical panoramic sound image instrument, the spherical panoramic sound image instrument comprises a microphone spherical array and a panoramic camera; the computing server comprises a processing module, a locking module, a multi-modal positioning module, an extraction module, a detection and recognition module;

[0032] The microphone spherical array is used for real-time acquisition of sound signals;

[0033] The processing module is used for processing the sound signals based on a HOA-SHT domain analysis framework to generate panoramic sound images;

[0034] The panoramic camera is used for acquiring air conditioner images;

[0035] The locking module is used for locking an air conditioner area according to the air conditioner images;

[0036] The multi-modal positioning module is used for multi-modal positioning of the air conditioner area through the panoramic sound images to obtain a target area;

[0037] The extraction module is used for sound signal extraction of the target area through multi-scale mel spectrum, and inputting the extracted sound signals into a beamformer of a neural network to obtain noise signals;

[0038] The detection and recognition module is used for detection and recognition of the noise signals based on an abnormality classifier of a convolutional neural network to obtain an air conditioner abnormal sound classification result.

[0039] Further, the processing module is specifically used for:

[0040] Pretreatment of the sound signals through Hanning window short-time Fourier transform to obtain pretreated sound signals;

[0041] Ball harmonic domain transformation of the pretreated sound signals to obtain 7-order surround sound field decomposition;

[0042] The 7th-order surround sound field decomposition is compensated by spherical harmonic coefficient energy to obtain a spherical harmonic domain feature.

[0043] The spherical harmonic domain feature is input into a beamformer to obtain a sound field.

[0044] The sound field is input into a panoramic beam space to obtain a spatial orientation feature.

[0045] The spatial orientation feature is superimposed with the air conditioner image to obtain the panoramic sound image.

[0046] In a third aspect, an electronic device is provided, including a memory and a processor, the memory storing a computer program executable on the processor, and the processor executes the computer program to implement the method described above.

[0047] In a fourth aspect, a computer readable medium having non-volatile program code executable by a processor is provided, and the program code causes the processor to execute the method described above.

[0048] The embodiments of the present application provide an air conditioner production line abnormal sound detection method and system, including: acquiring sound signals in real time through a microphone spherical array; processing the sound signals based on a HOA-SHT domain analysis framework to generate a panoramic sound image; acquiring an air conditioner image through a panoramic camera; locking an air conditioner area according to the air conditioner image; performing multi-modal positioning on the air conditioner area through the panoramic sound image to obtain a target area; extracting sound signals from the target area through a multi-scale Mel spectrum, and inputting the extracted sound signals into a beamformer of a neural network to obtain noise signals; detecting and identifying the noise signals based on a convolutional neural network-based anomaly classifier to obtain an air conditioner abnormal sound classification result; efficiently and accurately detecting air conditioner hanging machine abnormal sound, effectively overcoming the limitations of traditional manual detection methods, significantly improving production efficiency, reducing false detection rate and missed detection rate, and providing strong guarantee for production line quality.

[0049] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and achieved by means of the structures particularly pointed out in the description and the claims.

[0050] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the specific embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0052] Figure 1 The flow chart of the abnormal sound detection method of the air conditioner production line provided by the first embodiment of the present application;

[0053] Figure 2 The schematic diagram of the spherical panoramic sound image instrument provided by the first embodiment of the present application;

[0054] Figure 3 The schematic diagram of the abnormal sound detection process of the air conditioner production line provided by the first embodiment of the present application;

[0055] Figure 4 The schematic diagram of the abnormal sound detection system of the air conditioner production line provided by the second embodiment of the present application;

[0056] Figure 5 The schematic diagram of another abnormal sound detection system of the air conditioner production line provided by the second embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the present application will be described clearly and completely below in combination with the drawings. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0058] In the prior art, foreign matters such as dust, particulate matter, oil stains or other production residues may be attached to the surface of the wind wheel blade. The attachment of these foreign matters on the surface of the wind wheel blade may cause abnormal vibration of the blade during operation, thereby causing noise problems.

[0059] When the air conditioner is running, the rotation speed of the wind wheel blade is high, especially in the cooling or heating mode. The high-speed operation of the blade is easy to bring the attached foreign matters into the airflow, causing friction or collision between the blade and the foreign matters, and producing additional noise. This noise may be covered by environmental noise during the day, but when the environmental noise is low at night, the weak noise may also become abnormally obvious, seriously affecting the sleep quality and life experience of the user.

[0060] Therefore, how to effectively avoid the adhesion of foreign matters to the fan blades in the production process of the air conditioner hanging machine and how to reduce the blade noise problem caused by the foreign matters become technical problems that air conditioner manufacturing enterprises urgently need to solve. The air conditioner production line abnormal sound detection method and system provided in the application not only help improve the quality of air conditioner products, but also significantly improve the user experience, and have important practical significance and market value.

[0061] In order to facilitate the understanding of the present embodiment, the present embodiment will be described in detail below.

[0062] Embodiment one:

[0063] Figure 1 The flow chart of the air conditioner production line abnormal sound detection method provided in the present embodiment one.

[0064] With reference to Figure 1 , the method comprises the following steps:

[0065] Step S101, real-time acquisition of sound signals by a microphone spherical array;

[0066] Step S102, processing of the sound signals based on a HOA-SHT domain analysis framework to generate a panorama sound image;

[0067] Step S103, acquisition of an air conditioner image by a panoramic camera;

[0068] Step S104, locking of an air conditioner area according to the air conditioner image;

[0069] Step S105, multi-modal positioning of the air conditioner area by the panorama sound image to obtain a target area;

[0070] Step S106, sound signal extraction of the target area by a multi-scale mel spectrum, and input of the extracted sound signals into a neural network beamformer to obtain a noise signal;

[0071] Step S107, detection and identification of the noise signal based on a convolutional neural network abnormality classifier to obtain an air conditioner abnormal sound classification result.

[0072] In the present embodiment, the spherical panorama sound image instrument comprises a microphone spherical array and a panoramic camera; the spatial omnidirectional directivity of the microphone spherical array is utilized to effectively suppress noise interference. By deploying the microphone spherical array on the production line, sound signals are acquired in real time. At the same time, the panoramic camera captures the spatial information of the air conditioner, and in combination with a target detection model (YOLOv11), accurately judges the production line running state and positions the air conditioner position. Subsequently, the system extracts sound signals from the target area, and utilizes a neural network-based beamformer to accurately extract weak noise signals. Finally, the abnormality classifier detects and identifies the abnormal sound, realizing efficient and accurate detection of the air conditioner hanging machine abnormal sound.

[0073] Referring to Figure 2 , the spherical panoramic sound image instrument comprises a spherical microphone array and a panoramic camera, the spherical microphone array comprising an acoustic acquisition array, a visual auxiliary unit and an intelligent processing platform.

[0074] The main technical features are as follows: the sound field acquisition array is a spherical acoustic array constructed based on a 64-channel vector microphone, spatial sound field reconstruction is realized through a precise three-dimensional equiangular distribution algorithm, 7-order Ambisonics high-order surround sound field modeling is supported, and spatial isotropic directivity is possessed. The unique gradient phase compensation algorithm can effectively suppress multipath reflection and environmental noise, and in combination with adaptive beamforming technology, directional enhancement extraction of the air conditioning target sound source is realized, and the azimuth resolution reaches ±3°.

[0075] The spherical panoramic sound image instrument further comprises a main control unit, the main control unit adopting an FPGA Xilinx ZYNQ UltraScale+MPSoC platform, integrating real-time signal processing and logic control functions. The audio processing link adopts a Cirrus Logic CS5348 multi-channel ADC (24bit / 192kHz), dynamic range expansion processing (DRC 120dB) and a digital anti-aliasing filter (FIR 512-order).

[0076] The spherical panoramic sound image instrument further comprises a panoramic camera, the panoramic camera comprising 4 IMX477 cameras (12 million pixels, 60fps) and H.265 hardware encoding (4K@30fps). The timing system supports GPS / Beidou dual-mode time service (±50ns accuracy) and PTPv2 network clock synchronization. The system works in a 200MHz main clock domain, realizes synchronous acquisition of audio and video through a DMA architecture, audio streams are transmitted through a JESD204B interface, a TDM time division multiplexing mode is adopted, video data is transmitted through a MIPI CSI-2 interface, and YUV422 10bit quantization is adopted.

[0077] The system can realize target sound source separation in a complex sound field environment (signal-to-noise ratio≥-10dB) through acoustic-optical joint calibration (calibration error<0.1°) and intelligent noise reduction algorithm, and meets the demand of precise sound field analysis of air conditioning noise on a production line.

[0078] Further, the step S102 comprises the following steps:

[0079] In step S201, the sound signal is preprocessed through a Hanning window short-time Fourier transform to obtain a preprocessed sound signal.

[0080] In step S202, the preprocessed sound signal is subjected to a spherical harmonic domain transform to obtain a 7-order surround sound field decomposition.

[0081] Step S203, the 7th order ambisonics field decomposition is compensated by spherical harmonic coefficient energy to obtain a spherical harmonic domain feature;

[0082] Step S204, the spherical harmonic domain feature is input into a beamformer to obtain a sound field;

[0083] Step S205, the sound field is input into a panoramic beam space to obtain a spatial orientation feature;

[0084] Step S206, the spatial orientation feature is superimposed with an air conditioner image to obtain a panoramic sound image.

[0085] Specifically, the application adopts an acousto-optic fusion detection architecture, and implements a multi-modal abnormal acoustic detection method based on an HOA-SHT domain analysis framework. Referring to Figure 3 The algorithm first jointly represents the sound field, pre-processes the sound signal by Hanning window short-time Fourier transform (window length 512 points, overlap rate 75%), and obtains a 7th order ambisonics field decomposition (64 channel B-format encoding) by using spherical harmonic domain transformation (SHT); and the 7th order ambisonics field decomposition is compensated by spherical harmonic coefficient energy to obtain a spherical harmonic domain feature.

[0086] Subsequently, an improved target detection model (YOLOv11) is used for multi-modal positioning, and the target area is accurately locked by panoramic image visual guidance. Among them, the air conditioner images in multiple directions are acquired by a panoramic camera, the air conditioner images in multiple directions are spliced, and then the spliced images are input into the target detection model.

[0087] On this basis, the system uses a spherical harmonic wave domain beamformer to dynamically adjust a spatial filter set for continuous tracking. In the feature extraction stage, the system extracts the frequency domain features of the air conditioner signal by using a multi-scale mel spectrum, and combines a classifier based on a convolutional neural network and a hybrid attention mechanism to realize accurate detection and identification of air conditioner abnormal sound types. Finally, the system reports the air conditioner abnormal sound classification results to the production line workers in real time, and provides timely troubleshooting basis for them.

[0088] Further, step S104 includes the following steps:

[0089] Step S301, input the air conditioner image into the target detection model to obtain the spatial position of the air conditioner;

[0090] Step S302, lock the air conditioner area according to the spatial position of the air conditioner.

[0091] Further, the method further includes the following steps:

[0092] Step S401, monitor the running state of the production line in real time by using a panoramic camera;

[0093] Step S402, when the running state of the production line is a shutdown state, the audio and video collection is paused.

[0094] Further, the method further comprises the following steps:

[0095] Step S501, the air conditioner abnormal sound classification result is prompted through voice broadcast.

[0096] Further, the air conditioner abnormal sound classification result includes foreign matter adhesion, blade burr and sheet metal vibration.

[0097] Specifically, the panoramic camera can not only provide auxiliary positioning function for the sound field, but also be used for real-time monitoring of the running state of the production line (such as shutdown or running state). When it is detected that the running state of the production line is a shutdown state, the system will automatically pause the audio and video collection. In addition, the panoramic camera can capture air conditioner images, and determine the spatial position of the air conditioner by combining the target detection algorithm YOLOv11, so as to lock the target area through the panoramic sound image instrument, and extract and enhance the air conditioner wind wheel noise signal.

[0098] Compared with the traditional manual sound screening method, the application adopts innovative panoramic acoustic perception technology, combines sound-light fusion detection architecture, and significantly improves the anti-interference performance of the system. At the same time, by introducing the AI visual guidance mechanism of the improved YOLOv11 algorithm, the target area is quickly positioned and tracked, so as to efficiently extract the air conditioner noise signal.

[0099] On this basis, the application also adopts an advanced AI abnormal sound detection model to further optimize the performance indicators of abnormal sound detection. Through these technical means, the application effectively overcomes the limitations of the traditional manual detection method, significantly improves the production efficiency, reduces the false detection rate and the missed detection rate, and provides a strong guarantee for the production line quality.

[0100] Embodiment two:

[0101] Figure 4 The air conditioner production line abnormal sound detection system provided for embodiment two of the application.

[0102] Referring to Figure 4 , the system includes a computing server and a spherical panoramic sound image instrument, and the spherical panoramic sound image instrument includes a microphone spherical array and a panoramic camera; referring to Figure 5 , the computing server includes a processing module, a locking module, a multi-modal positioning module, an extraction module, a detection and identification module;

[0103] The microphone spherical array is used for real-time acquisition of sound signals;

[0104] The processing module is used for processing the sound signals based on the HOA-SHT domain analysis framework to generate panoramic sound images;

[0105] a panoramic camera, configured to acquire an air conditioner image;

[0106] a locking module, configured to lock an air conditioner area according to the air conditioner image;

[0107] a multi-modal positioning module, configured to perform multi-modal positioning on the air conditioner area through the panoramic sound image to obtain a target area;

[0108] an extraction module, configured to perform sound signal extraction on the target area through a multi-scale mel spectrum, and input the extracted sound signal into a beamformer of a neural network to obtain a noise signal;

[0109] a detection and recognition module, configured to perform detection and recognition on the noise signal based on a convolutional neural network-based anomaly classifier to obtain an air conditioner abnormal sound classification result.

[0110] Specifically, in the Figure 4 , the system comprises a computing server and a spherical panoramic sound image instrument, the spherical panoramic sound image instrument comprises a microphone spherical array and a panoramic camera, and is installed above a production line.

[0111] The system further comprises a label scanning machine, the air conditioner ID is acquired through the label scanning machine, and is associated with the sound signal. Finally, the computing server performs detection and recognition on the noise signal to obtain an air conditioner abnormal sound classification result; the air conditioner abnormal sound classification result is prompted to a production line worker through voice broadcast, so that the worker can timely troubleshoot and handle a faulty air conditioner.

[0112] Further, the processing module is specifically configured to:

[0113] perform preprocessing on the sound signal through a Hanning window short-time Fourier transform to obtain preprocessed sound signal;

[0114] perform spherical harmonic domain transformation on the preprocessed sound signal to obtain 7-order surround sound field decomposition;

[0115] perform spherical harmonic coefficient energy compensation on the 7-order surround sound field decomposition to obtain spherical harmonic domain features;

[0116] perform beamforming on the spherical harmonic domain features to obtain a sound field;

[0117] perform panoramic beam space transformation on the sound field to obtain spatial orientation features of each space;

[0118] superimpose the spatial orientation features of each space and the air conditioner image to obtain a panoramic sound image.

[0119] The embodiment of the present application also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the air conditioner production line abnormal sound detection method provided by the above-mentioned embodiment when executing the computer program.

[0120] The embodiment of the present application further provides a computer readable medium with non-volatile program codes executable by a processor, the computer readable medium storing a computer program, and the computer program being executed by the processor to perform the steps of the abnormal sound detection method of the air conditioner production line.

[0121] The computer program product provided by the embodiment of the present application comprises a computer readable storage medium storing program codes, and the program codes comprise instructions for performing the method described in the foregoing method embodiment. For details, refer to the method embodiment, which will not be repeated here.

[0122] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

[0123] In addition, in the description of the embodiment of the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connection" and "connecting" should be understood in a broad sense, for example, can be fixedly connected, or can be detachably connected, or integrally connected; can be mechanically connected, or can be electrically connected; can be directly connected, or can be indirectly connected through an intermediate medium; can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0124] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the part of the technical solutions that essentially contribute to the prior art can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0125] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", "third" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0126] Finally, it should be noted that the above-described embodiments are only specific implementations of the present application, which are used to illustrate the technical solutions of the present application, and are not limiting. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features, within the technical scope disclosed by the present application. Such modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An abnormal sound detection method of an air conditioner production line, characterized by, The method comprises: Acquire acoustic signals in real time through a spherical array of microphones; Processing the acoustic signal based on the HOA-SHT domain analysis framework to generate a panoramic sound image; Acquire air conditioning images through panoramic cameras; locking the air-conditioning area according to the air-conditioning image; Perform multimodal positioning of the air-conditioned area using the panoramic audio and video to obtain a target area; Extracting acoustic signals from the target area using a multi-scale Mel spectrum, and inputting the extracted acoustic signals into a beamformer of a neural network to obtain a noise signal; The abnormality classifier based on the convolutional neural network detects and identifies the noise signal to obtain the air conditioner abnormal sound classification result.

2. The abnormal sound detection method of an air conditioning production line according to claim 1, characterized in that, The acoustic signal is processed based on the HOA-SHT domain analysis framework to generate a panoramic sound image, including: Preprocessing the acoustic signal by using a Hanning window short-time Fourier transform to obtain a preprocessed acoustic signal; Transforming the preprocessed acoustic signal through spherical harmonics to obtain a 7th-order surround sound field decomposition; Decomposing the 7th-order surround sound field and performing spherical harmonic coefficient energy compensation to obtain spherical harmonic domain characteristics; Passing the spherical harmonic domain characteristics through a beamformer to obtain a sound field; Passing the sound field through the panoramic beam space to obtain various spatial orientation features; The various spatial orientation features and the air-conditioning image are superimposed to obtain the panoramic sound image.

3. The abnormal sound detection method of an air conditioning production line according to claim 1, characterized by, Locking the air-conditioning area according to the air-conditioning image includes: Inputting the air conditioner image into a target detection model to obtain the spatial position of the air conditioner; The air-conditioning area is locked according to the spatial position of the air-conditioner.

4. The abnormal sound detection method of an air conditioning production line according to claim 1, characterized in that, The method further comprises: Real-time monitoring of the operating status of the production line through the panoramic camera; When the production line is in a shutdown state, audio and video collection is controlled to be paused.

5. The abnormal sound detection method of an air conditioning production line according to claim 1, characterized in that, The method further comprises: The classification result of the abnormal sound of the air conditioner is prompted by voice broadcast.

6. The abnormal sound detection method of an air conditioning production line according to claim 1, characterized in that, The classification results of the abnormal sound of the air conditioner include foreign matter adhesion, blade burrs and sheet metal vibration.

7. An abnormal sound detection system for an air-conditioning production line, characterized in that: The system includes a computing server and a spherical panoramic audio and video camera, wherein the spherical panoramic audio and video camera includes a spherical microphone array and a panoramic camera; the computing server includes a processing module, a locking module, a multimodal positioning module, an extraction module, and a detection and recognition module; The microphone spherical array is used to acquire acoustic signals in real time; The processing module is used to process the acoustic signal based on the HOA-SHT domain analysis framework to generate a panoramic sound image; The panoramic camera is used to obtain air conditioning images; The locking module is configured to lock the air-conditioning area according to the air-conditioning image; The multimodal positioning module is used to perform multimodal positioning of the air-conditioned area through the panoramic audio and video to obtain a target area; The extraction module is used to extract the acoustic signal of the target area through the multi-scale Mel spectrum, and input the extracted acoustic signal into the beamformer of the neural network to obtain a noise signal; The detection and recognition module is used to detect and recognize the noise signal based on the abnormality classifier of the convolutional neural network to obtain the classification result of the abnormal sound of the air conditioner.

8. The air conditioning line abnormal sound detection system of claim 7, wherein The processing module is specifically used for: Preprocessing the acoustic signal by using a Hanning window short-time Fourier transform to obtain a preprocessed acoustic signal; The preprocessed sound signal is subjected to a spherical harmonic domain transformation to obtain a 7th order surround sound field decomposition; The 7th order surround sound field decomposition is subjected to spherical harmonic coefficient energy compensation to obtain a spherical harmonic domain feature; The spherical harmonic domain feature is subjected to a beamformer to obtain a sound field; The sound field is subjected to a panoramic beam space to obtain various spatial orientation features; The various spatial orientation features and the air conditioning image are superimposed to obtain the panoramic sound image.

9. An electronic device comprising a memory, a processor, said memory having stored thereon a computer program operable to run on said processor, characterized in that, The processor executes the computer program to implement the method of any one of claims 1 to 6.

10. A computer readable medium having non-transitory program code executable by a processor, the program code comprising instructions for: The program code causes the processor to execute the method of any one of claims 1 to 6.

Images