An abnormal sound detection method, device, equipment, storage medium and product

Abstract

The application provides an abnormal sound detection method, device, equipment, storage medium and product. The abnormal sound detection method comprises the following steps: acquiring original audio collected during abnormal sound detection; determining a time-frequency matrix corresponding to the original audio; the time-frequency matrix is a matrix for describing the original audio in a time dimension and a frequency dimension; detecting constant frequency information in the time-frequency matrix; and determining an abnormal sound detection result corresponding to the original audio based on the constant frequency information and an abnormal sound detection rule. Through the application, the accuracy of abnormal sound detection can be improved.

Description

Technical Field

[0001] This application relates to the field of abnormal sound detection technology, and in particular to an abnormal sound detection method, apparatus, equipment, storage medium and product. Background Technology

[0002] During the production of electric vehicles, some components may emit abnormal noises at a constant frequency. Some of these abnormal noises are loud, while others are soft.

[0003] In related technologies, a decibel threshold is set. After determining the decibel value obtained by performing a Fast Fourier Transform (FFT) on the frequency corresponding to the sound emitted by the component, this decibel value is compared with the decibel threshold. If the decibel value is greater than the threshold, an abnormal sound is determined to exist; otherwise, no abnormal sound is determined to exist. However, this method has a very strict threshold setting, requiring a completely consistent detection environment. Furthermore, when the abnormal sound is relatively quiet, the change in decibel value is also relatively small, resulting in a very high false detection rate, thus reducing the accuracy of abnormal sound detection. Summary of the Invention

[0004] This application provides an abnormal sound detection method, apparatus, device, storage medium, and product, which can improve the accuracy of abnormal sound detection.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides an abnormal sound detection method, the method comprising:

[0007] Acquire the raw audio collected during abnormal sound detection;

[0008] Determine the time-frequency matrix corresponding to the original audio; the time-frequency matrix is ​​a matrix that describes the original audio in terms of time and frequency dimensions;

[0009] Detect constant frequency information in the time-frequency matrix;

[0010] Based on the constant frequency information and the abnormal sound detection rules, the abnormal sound detection result corresponding to the original audio is determined.

[0011] In the above scheme, detecting constant frequency information in the time-frequency matrix includes:

[0012] In the time-frequency matrix, constant frequency edge detection is performed to obtain a detection image;

[0013] The detected image is subjected to connectivity analysis and filtering to obtain a first image;

[0014] Line detection is performed on the first image to obtain the constant frequency information.

[0015] In the above scheme, the step of performing constant frequency edge detection in the time-frequency matrix to obtain a detection image includes:

[0016] Obtain the reference value and the amplitude of each matrix parameter in the time-frequency matrix;

[0017] The amplitude is adjusted based on the reference value to obtain the target matrix;

[0018] Edge detection is performed based on the matrix parameters in the target matrix to obtain the detected image.

[0019] In the above scheme, the step of performing line detection on the first image to obtain the constant frequency information includes:

[0020] Obtain the hyperparameters for line detection;

[0021] Based on the hyperparameters, line detection is performed on the first image to obtain the constant frequency information.

[0022] In the above scheme, determining the time-frequency matrix corresponding to the original audio includes:

[0023] The original audio is subjected to a short-time Fourier transform to obtain the time-frequency matrix.

[0024] In the above scheme, the step of performing a short-time Fourier transform on the original audio to obtain the time-frequency matrix includes:

[0025] The original audio is segmented according to the first segmentation step size and the first window length to obtain multiple audio segments;

[0026] The time-frequency matrix is ​​obtained by performing a short-time Fourier transform on the multiple audio segments.

[0027] In the above scheme, determining the abnormal sound detection result corresponding to the original audio based on the constant frequency information and the abnormal sound detection rules includes:

[0028] If the frequency corresponding to the constant frequency information is within the frequency range of the abnormal sound detection rule, and the start time corresponding to the constant frequency information is the same as the start time in the abnormal sound detection rule, then the abnormal sound detection result is determined to be that the original audio is an abnormal sound.

[0029] If the frequency corresponding to the constant frequency information is not within the frequency range of the abnormal sound detection rule, or if the start time corresponding to the constant frequency information is different from the start time in the abnormal sound detection rule, the abnormal sound detection result is determined to be that the original audio is not an abnormal sound.

[0030] This application provides an abnormal sound detection device, including:

[0031] The acquisition unit is used to acquire the raw audio collected during abnormal sound detection;

[0032] A determining unit is used to determine the time-frequency matrix corresponding to the original audio; the time-frequency matrix is ​​a matrix describing the original audio in terms of time and frequency dimensions; and to determine the abnormal sound detection result corresponding to the original audio based on constant frequency information and abnormal sound detection rules.

[0033] A detection unit is used to detect the constant frequency information in the time-frequency matrix.

[0034] In the above scheme, the device further includes a processing unit;

[0035] The detection unit is used to perform constant frequency edge detection in the time-frequency matrix to obtain a detection image; and to perform line detection on the first image to obtain the constant frequency information.

[0036] The processing unit is used to perform connectivity analysis and filtering on the detected image to obtain a first image.

[0037] In the above scheme, the device further includes an adjustment unit;

[0038] The acquisition unit is used to acquire the reference value and the amplitude of each matrix parameter in the time-frequency matrix;

[0039] The adjustment unit is used to adjust the amplitude based on the reference value to obtain the target matrix;

[0040] The processing unit is used to perform edge detection processing based on the matrix parameters in the target matrix to obtain the detected image.

[0041] In the above scheme, the acquisition unit is used to acquire the hyperparameters of line detection;

[0042] The detection unit is used to perform line detection on the first image based on the hyperparameters to obtain the constant frequency information.

[0043] In the above scheme, the processing unit is used to perform short-time Fourier transform processing on the original audio to obtain the time-frequency matrix.

[0044] In the above scheme, the processing unit is used to segment the original audio according to the first segmentation step size and the first window length to obtain multiple audio segments; and to perform short-time Fourier transform processing on the multiple audio segments to obtain the time-frequency matrix.

[0045] In the above scheme, the determining unit is used to determine that the original audio is an abnormal sound when the frequency corresponding to the constant frequency information is within the frequency range of the abnormal sound detection rule and the start time corresponding to the constant frequency information is the same as the start time in the abnormal sound detection rule; and to determine that the original audio is not an abnormal sound when the frequency corresponding to the constant frequency information is not within the frequency range of the abnormal sound detection rule, or the start time corresponding to the constant frequency information is different from the start time in the abnormal sound detection rule.

[0046] This application embodiment provides an abnormal sound detection device, the abnormal sound detection device comprising:

[0047] Memory is used to store executable instructions for a computer;

[0048] The processor, when executing computer-executable instructions stored in the memory, implements the abnormal sound detection method provided in the embodiments of this application.

[0049] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the abnormal sound detection method provided in this application.

[0050] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the abnormal sound detection method provided in this application.

[0051] The embodiments of this application have the following beneficial effects: When the abnormal sound detection device acquires the original audio collected during abnormal sound detection, it determines the time-frequency matrix corresponding to the original audio and detects constant frequency information in the time-frequency matrix. Based on the constant frequency information and the abnormal sound detection rules, it determines the abnormal sound detection result corresponding to the original audio. It is not necessary to compare the decibel value obtained after performing FFT transformation on the frequency corresponding to the sound with the decibel threshold. The abnormal sound detection result corresponding to the original audio can be determined, avoiding false detection results in cases of inconsistent detection environments and improving the accuracy of abnormal sound detection. Attached Figure Description

[0052] Figure 1This is a flowchart of an abnormal sound detection method provided in an embodiment of this application;

[0053] Figure 2 This is an exemplary HIIS noise spectrum diagram provided in an embodiment of this application;

[0054] Figure 3 This is a schematic diagram of an exemplary Canny edge detection process provided in an embodiment of this application;

[0055] Figure 4 This is a schematic diagram of an exemplary abnormal sound detection process provided in an embodiment of this application;

[0056] Figure 5 This is a schematic diagram illustrating an exemplary abnormal sound detection result provided in an embodiment of this application;

[0057] Figure 6 This is a schematic diagram of the structural composition of an abnormal sound detection device provided in an embodiment of this application;

[0058] Figure 7 This is a schematic diagram of the composition structure of an abnormal sound detection device provided in an embodiment of this application. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0061] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0062] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0063] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant national laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0064] This application provides an abnormal sound detection method, which is applied to an abnormal sound detection device. Figure 1 A flowchart of an abnormal sound detection method provided in an embodiment of this application is shown below. Figure 1 As shown, abnormal sound detection methods may include:

[0065] S101. Obtain the original audio collected during abnormal sound detection.

[0066] The abnormal sound detection method provided in this application embodiment is applicable to scenarios where abnormal sound detection is performed on the acquired raw audio.

[0067] In the embodiments of this application, the abnormal sound detection device can be implemented in various forms. For example, the abnormal sound detection device described in this application may include devices such as a central processing unit, an in-vehicle controller, a cloud platform, and a server. The specific abnormal sound detection device can be determined according to the actual situation, and the embodiments of this application do not limit it in this regard.

[0068] For example, the abnormal sound detection method can be applied to detect whether there is a constant-frequency abnormal noise during the operation of the air conditioner in an electric vehicle. Specifically, it can be used to detect whether the air conditioner of an electric vehicle has a HIIS (High-Intensity Sound Indicator) abnormal noise during the electric vehicle production process.

[0069] In this embodiment, when the air conditioner of an electric vehicle is started, the refrigerant containing air bubbles flows through the expansion valve of the air conditioning unit, generating a high-frequency sound that is then transmitted to the passenger compartment. The spectrum diagram of the air conditioner exhibiting a constant-frequency HIIS abnormal noise is shown below. Figure 2 As shown, the HIIS abnormal noise mainly occurs in two frequency bands: 2500-3000 Hz and 4100-4300 Hz. Figure 2 The frequency line enclosed in a rectangular frame.

[0070] In this embodiment of the application, a constant frequency abnormal sound detection switch can be set on the display screen of the electric vehicle. After the constant frequency abnormal sound detection switch is triggered on the display screen to start abnormal sound detection, the sound of the air conditioner can be collected using the vehicle microphone to obtain the original audio.

[0071] It should be noted that a sampling frequency of 16kHz can be used to collect the sound of the air conditioner to obtain the original audio; other sampling frequencies can also be used to collect the sound of the air conditioner to obtain the original audio; the specific sampling frequency of the original audio can be determined according to the actual situation, and this application embodiment does not limit it in this regard.

[0072] In this embodiment, the original audio can be obtained from the device that collects the original audio, from the database that stores the original audio, or through other means. The specific method of obtaining the original audio can be determined according to the actual situation, and this embodiment does not limit it.

[0073] In the embodiments of this application, the original audio can be a sound signal with only a time dimension.

[0074] S102. Determine the time-frequency matrix corresponding to the original audio.

[0075] In this embodiment of the application, after the abnormal sound detection device acquires the original audio collected during abnormal sound detection, it determines the time-frequency matrix corresponding to the original audio.

[0076] It should be noted that the time-frequency matrix can be a matrix that describes the original audio in terms of both time and frequency dimensions.

[0077] In the embodiments of this application, the method for determining the time-frequency matrix corresponding to the original audio can be determined according to the actual situation, and the embodiments of this application do not limit it.

[0078] In this embodiment of the application, the process by which the abnormal sound detection device determines the time-frequency matrix corresponding to the original audio includes: performing a short-time Fourier transform on the original audio to obtain the time-frequency matrix.

[0079] In this embodiment of the application, the parameter information of the Short-Time Fourier Transform (STFT) can be obtained, and the original audio can be processed by the Short-Time Fourier Transform based on the parameter information to obtain the time-frequency matrix.

[0080] It should be noted that the parameter information of the short-time Fourier transform includes n_fft and hop_size, etc.

[0081] For example, n_fft can be 2048 and hop_size can be 512.

[0082] In this embodiment, the parameter information of the short-time Fourier transform can be obtained from a database; it can also be obtained from other devices; or it can be obtained from information input by the user. The specific method of obtaining the parameter information of the short-time Fourier transform can be determined according to the actual situation, and this embodiment does not limit it.

[0083] In this embodiment of the application, the original audio can be preprocessed to obtain preprocessed audio; then, the preprocessed audio can be subjected to short-time Fourier transform to obtain a time-frequency matrix.

[0084] It should be noted that preprocessing can be segmented processing, filtering processing, or noise reduction processing. The specific preprocessing method can be determined according to the actual situation, and this application embodiment does not limit it.

[0085] In this embodiment of the application, the abnormal sound detection device can first acquire the original audio collected during the detection of abnormal sound (such as the abnormal sound of HIIS in an air conditioner); then perform short-time Fourier transform processing on the original audio to obtain a time-frequency matrix describing the original audio in terms of time and frequency dimensions; then detect constant frequency information in the time-frequency matrix; and finally determine the abnormal sound detection result corresponding to the original audio based on the constant frequency information and the abnormal sound detection rule.

[0086] Understandably, using short-time Fourier transform to directly perform mathematical transformations on the original audio improves the speed of processing the original audio.

[0087] In this embodiment of the application, the process of the abnormal sound detection device performing short-time Fourier transform processing on the original audio to obtain a time-frequency matrix includes: segmenting the original audio according to a first segmentation step size and a first window length to obtain multiple audio segments; and performing short-time Fourier transform processing on the multiple audio segments to obtain a time-frequency matrix.

[0088] In this embodiment, the first segment step size can be configured step size information, information transmitted from other devices to the abnormal sound detection device, or information obtained by the abnormal sound detection device through other means. The specific way in which the abnormal sound detection device obtains the first segment step size can be determined according to the actual situation, and this embodiment does not limit it.

[0089] In this embodiment, the first window length can be configured length information, information transmitted from other devices to the abnormal sound detection device, or information obtained by the abnormal sound detection device through other means. The specific way the abnormal sound detection device obtains the first window length can be determined according to the actual situation, and this embodiment does not limit it.

[0090] For example, the first segmentation step size can be 5S, and the first window length can be 10S. Then, the original audio is segmented according to the first segmentation step size and the first window length, and the resulting audio segments can be: the first audio segment is 1S-10S, the second audio segment is 5-15S, the third audio segment is 10-20S, the fourth audio segment is 15-25S; and so on.

[0091] In the embodiments of this application, short-time Fourier transform processing can be performed on multiple audio segments to obtain a time-frequency matrix.

[0092] In this embodiment, the abnormal sound detection device can first acquire the original audio collected during abnormal sound (such as the abnormal noise of an air conditioner HIIS) detection; then, it can segment the original audio according to a first segmentation step size (such as 5S) and a first window length (such as 10S) to obtain multiple audio segments; and perform short-time Fourier transform processing on the multiple audio segments to obtain a time-frequency matrix describing the original audio in terms of time and frequency dimensions; then, it can detect constant frequency information in the time-frequency matrix; and finally, based on the constant frequency information and the abnormal sound detection rule, it can determine the abnormal sound detection result corresponding to the original audio.

[0093] Understandably, the original audio is first segmented according to the first segment step size and the first window length to obtain multiple audio segments. This allows for simultaneous short-time Fourier transform processing of multiple audio segments to obtain the time-frequency matrix, thus improving the efficiency of obtaining the time-frequency matrix.

[0094] S103. Detect constant frequency information in the time-frequency matrix.

[0095] In this embodiment of the application, after the abnormal sound detection device determines the time-frequency matrix corresponding to the original audio, it detects constant frequency information in the time-frequency matrix.

[0096] It should be noted that constant frequency information can be audio information in the time-frequency matrix whose frequency remains constant over time. For example, audio with a frequency that remains at 2600Hz over time.

[0097] In the embodiments of this application, the number of constant frequency information in the time-frequency matrix can be one or more. The specific number of constant frequency information in the time-frequency matrix can be determined according to the actual situation, and the embodiments of this application do not limit this.

[0098] In this embodiment of the application, the process of the abnormal sound detection device detecting constant frequency information in a time-frequency matrix includes: performing constant frequency edge detection in the time-frequency matrix to obtain a detection image; performing connectivity analysis filtering on the detection image to obtain a first image; and performing line detection on the first image to obtain constant frequency information.

[0099] In this embodiment, constant frequency edge detection can be performed in the time-frequency matrix using the Canny edge detection method to obtain a detection image; alternatively, other edge detection methods can be used to perform constant frequency edge detection in the time-frequency matrix to obtain a detection image. The specific method of performing constant frequency edge detection in the time-frequency matrix to obtain a detection image can be determined according to the actual situation, and this embodiment does not limit it.

[0100] In this embodiment, the detection image is specifically a binary black-and-white image containing constant frequency information, meaning that the image value of the constant frequency information in the detection image is different from the image value of the non-constant frequency information in the detection image. For example, the image value of the constant frequency information in the detection image is 1, and the image value of the non-constant frequency information in the detection image is 0; or the image value of the constant frequency information in the detection image is 0, and the image value of the non-constant frequency information in the detection image is 1.

[0101] It should be noted that the detection image obtained after constant frequency edge detection is a binary image. This binary image includes many edge features, such as low-frequency noise areas, which may be detected as a large number of edges, causing interference to subsequent data processing. To remove such large noise edges, connectivity analysis can be used to filter the detection image to obtain constant frequency information.

[0102] In this embodiment, connectivity analysis parameters can be obtained first; based on the connectivity analysis parameters, the detection image can be filtered by connectivity analysis to obtain a first image.

[0103] For example, the connectedComponentsWithStats function can be used to perform connectivity analysis filtering on the detected image based on connectivity analysis parameters to obtain the first image.

[0104] In this embodiment, connectivity analysis filtering technology can label connected regions in a binary image and calculate statistical information for each connected region. The principle of connectivity analysis filtering technology is based on image morphology and connected region analysis, and its main steps include: image preprocessing, connected region labeling, statistical information calculation, and outputting results.

[0105] The image preprocessing part requires ensuring that the input image is a binary image, that is, it contains only two pixel values, 0 and 1, where 0 represents the background and 1 represents the foreground. Then, methods such as thresholding and edge detection are used to convert the binary image into a grayscale image.

[0106] The connected component labeling process scans the image starting from the top left corner, assigning a unique label value to each foreground pixel. If a foreground pixel has the same label value as its neighboring pixels (4-neighborhood or 8-neighborhood), they are merged into the same connected component. After the scan is complete, each connected component will have a unique label value.

[0107] The statistical information calculation section calculates the area (the number of pixels contained in the connected region), the boundary rectangle (the smallest rectangle that can contain the entire connected region, returning the coordinates of the top left corner, width, and height), the centroid (the coordinates of the geometric center of the connected region), and the list of pixel values ​​(a list of all pixel values ​​of the connected region) for each connected region.

[0108] The output section is used to output the number of connected components (including the background), the labeled image (the same size as the input image, where the value of each pixel represents the label of the connected component it belongs to), the statistical information matrix (containing statistical information for each connected component, with each row representing a connected component and columns corresponding to different statistical information), and the centroid matrix (containing the centroid coordinates of each connected component).

[0109] In this embodiment, constant frequency information in the first image can be identified by Hough direct connection detection, or other methods can be used to identify constant frequency information in the first image; the specific method can be determined according to the actual situation, and this embodiment does not limit it.

[0110] In this embodiment, the method of performing line detection on the first image to obtain constant frequency information can be determined according to the actual situation, and this embodiment does not limit it.

[0111] In this embodiment, the abnormal sound detection device can first acquire the original audio collected during abnormal sound (such as the abnormal noise of an air conditioner HIIS) detection; then, it can segment the original audio according to a first segmentation step size (such as 5S) and a first window length (such as 10S) to obtain multiple audio segments; and perform short-time Fourier transform processing on the multiple audio segments to obtain a time-frequency matrix describing the original audio in terms of time and frequency dimensions; then, it can perform constant frequency edge detection (such as Canny edge detection) on the time-frequency matrix to obtain a detection image; perform connectivity analysis filtering processing on the detection image to obtain a first image; perform line detection (such as Hough line detection) on the first image to obtain constant frequency information; and finally, based on the constant frequency information and the abnormal sound detection rule, determine the abnormal sound detection result corresponding to the original audio.

[0112] It is understandable that by performing constant frequency edge detection on the time-frequency matrix to obtain the detection image, and using connectivity analysis to filter out interference information that is unrelated to the constant frequency information, accurate constant frequency information can be obtained after performing line detection on the first image, thus improving the accuracy of determining the constant frequency information.

[0113] In this embodiment of the application, the abnormal sound detection device performs constant frequency edge detection in a time-frequency matrix to obtain a detection image, including: acquiring reference values ​​and the amplitude of each matrix parameter in the time-frequency matrix; adjusting the amplitude based on the reference values ​​to obtain a target matrix; and performing edge detection processing based on the matrix parameters in the target matrix to obtain a detection image.

[0114] In this embodiment of the application, an edge detection threshold can be obtained, and edge detection processing can be performed on the matrix parameters in the target matrix based on the detection threshold to obtain a detection image.

[0115] In this embodiment, the detection threshold can be obtained from a database, from user input, or from other devices. The specific method for obtaining the edge detection threshold can be determined according to the actual situation, and this embodiment does not limit it.

[0116] For example, the detection threshold can be a dual threshold of Canny edge detection. For example, the dual threshold can be 10 and 25, or it can be other parameter values. The specific parameter values ​​of the dual threshold can be determined according to the actual situation, and this application embodiment does not limit them.

[0117] In the embodiments of this application, such as Figure 3 As shown, Canny edge detection includes steps such as frequency normalization, Gaussian filtering, gradient and direction calculation, non-maximum suppression, and double thresholding. The specific Canny edge detection method is described in accordance with existing technologies and will not be elaborated upon here.

[0118] In this embodiment, the time-frequency matrix is ​​a matrix obtained after STFT transformation. The unit of each matrix parameter in the time-frequency matrix is ​​amplitude. The amplitude unit is adjusted to decibel to obtain the target matrix.

[0119] In this embodiment of the application, the amplitude of each matrix parameter in the time-frequency matrix can be obtained from the time-frequency matrix.

[0120] In this embodiment, the reference value can be an information value configured in the abnormal sound detection device, an information value transmitted to the abnormal sound detection device from other devices, or an information value obtained by the abnormal sound detection device through other means. The specific way the abnormal sound detection device obtains the reference value can be determined according to the actual situation.

[0121] In this embodiment, the target matrix is ​​obtained by adjusting the amplitude based on the reference value as shown in formula (1):

[0122] dB=10*log10(amp / ref) (1)

[0123] It should be noted that amp represents the amplitude of each matrix parameter in the time-frequency matrix, ref represents the reference value, and dB represents the adjusted value corresponding to each matrix parameter in the time-frequency matrix. The target matrix can be obtained based on the adjusted values ​​corresponding to each matrix parameter in the time-frequency matrix.

[0124] For example, ref can be 200000.

[0125] In this embodiment, the abnormal sound detection device can first acquire the original audio collected during abnormal sound (such as the abnormal noise of an air conditioner HIIS) detection; then, it can segment the original audio according to a first segmentation step size (such as 5S) and a first window length (such as 10S) to obtain multiple audio segments; and perform short-time Fourier transform processing on the multiple audio segments to obtain a time-frequency matrix that describes the original audio in terms of time and frequency dimensions; then, it can acquire a reference value and the amplitude of each matrix parameter in the time-frequency matrix; adjust the amplitude based on the reference value (such as adjusting the amplitude to decibels) to obtain a target matrix; perform edge detection (such as Canny edge detection) based on the matrix parameters in the target matrix to obtain a detection image; perform connectivity analysis filtering processing on the detection image to obtain a first image; perform line detection (such as Hough line detection) on the first image to obtain constant frequency information; and finally, determine the abnormal sound detection result corresponding to the original audio based on the constant frequency information and the abnormal sound detection rule.

[0126] It is understandable that by adjusting the energy value of the matrix parameters in the time-frequency matrix, i.e. adjusting the amplitude based on the reference value, the target matrix is ​​obtained. This target matrix is ​​an image that meets the requirements of the Canny edge detection processing format, which makes it easier for Canny edge detection to identify constant frequency information in the image. Therefore, it is not necessary to compare the decibel value obtained after performing FFT transformation on the frequency corresponding to the sound with the decibel threshold, avoiding false detection results in the case of inconsistent detection environment, and improving the accuracy of abnormal sound detection.

[0127] In this embodiment of the application, the process of the abnormal sound detection device performing line detection on the first image to obtain constant frequency information includes: acquiring hyperparameters for line detection; and performing line detection on the first image based on the hyperparameters to obtain constant frequency information.

[0128] In this embodiment, hyperparameters for line detection can be obtained from a database, from other devices, or through other means. The specific method for obtaining hyperparameters for line detection can be determined according to the actual situation, and this embodiment does not limit it.

[0129] In this embodiment, if the line detection is specifically Hough line detection, the hyperparameters of Hough line detection may include distance resolution (rho) in Hough space, angular resolution (theta) in Hough space, accumulator threshold (thresho ld), minimum line length (minLineLength), and maximum line segment distance (maxLineGap), etc. The specific hyperparameters of Hough line detection can be determined according to the actual situation, and this embodiment does not limit them.

[0130] It should be noted that the distance resolution in Hough space is measured in pixels. A smaller distance resolution value in Hough space results in higher detection accuracy, but also greater computational complexity. For example, the distance resolution value in Hough space in this application can be 4.

[0131] It should be noted that the angular resolution in Hough space is measured in radians. A smaller angular resolution value in Hough space results in higher detection accuracy, but also a greater computational burden. For example, the angular resolution value in Hough space in this application could be np.pi / 90.

[0132] It should be noted that only lines that receive a sufficient number of votes (greater than the accumulator threshold) will be detected. A higher accumulator threshold results in fewer detected lines, but can filter out some noise. For example, the accumulator threshold in this application can be 220.

[0133] It should be noted that the minimum line length is in pixels. Line segments shorter than this minimum line length will be discarded. The minimum line length is used to filter out short line segment noise. For example, the minimum line length in this application can be 40.

[0134] It should be noted that the maximum line segment distance is in pixels. When the distance between two line segments is less than this maximum line segment distance, they are considered to be part of the same straight line. The maximum line segment distance is used to connect broken line segments. For example, the maximum line segment distance in this application can be 20.

[0135] In this embodiment, the Hough line detection process involves transforming the line equation in the Cartesian coordinate system to the polar coordinate system, and substituting the coordinates of each pixel into the obtained equation to obtain the corresponding line parameters, thereby obtaining constant frequency information.

[0136] In this embodiment, the abnormal sound detection device can first acquire the original audio collected during abnormal sound (such as the abnormal noise of an air conditioner HIIS) detection; then, it can segment the original audio according to a first segmentation step size (e.g., 5S) and a first window length (e.g., 10S) to obtain multiple audio segments; and perform short-time Fourier transform processing on the multiple audio segments to obtain a time-frequency matrix that describes the original audio in terms of time and frequency dimensions; then, it can acquire reference values ​​and the amplitude of each matrix parameter in the time-frequency matrix; adjust the amplitude based on the reference values ​​(e.g., adjust the amplitude to decibels) to obtain the target matrix; and based on the target... The matrix parameters in the matrix are processed for edge detection (such as Canny edge detection) to obtain the detection image; the detection image is then processed for connectivity analysis and filtering to obtain the first image; hyperparameters for line detection (such as Hough line detection hyperparameters, including distance resolution in Hough space, angular resolution in Hough space, accumulator threshold, minimum line length, and maximum line segment distance, etc.) are obtained; line detection (such as Hough line detection) is performed on the first image based on the hyperparameters to obtain constant frequency information; finally, based on the constant frequency information and the abnormal sound detection rule, the abnormal sound detection result corresponding to the original audio is determined.

[0137] It is understandable that by setting hyperparameters for line detection, line detection can be performed on the first image based on these hyperparameters to identify constant frequency information in the first image. Thus, based on this constant frequency information and the abnormal sound detection rules, the abnormal sound detection result corresponding to the original audio can be determined. It is not necessary to compare the decibel value obtained after performing FFT transformation on the frequency corresponding to the sound with the decibel threshold, thus avoiding false detection results in cases of inconsistent detection environments and improving the accuracy of abnormal sound detection.

[0138] S104. Based on constant frequency information and abnormal sound detection rules, determine the abnormal sound detection result corresponding to the original audio.

[0139] In this embodiment of the application, after detecting constant frequency information in the time-frequency matrix, the abnormal sound detection device determines the abnormal sound detection result corresponding to the original audio based on the constant frequency information and the abnormal sound detection rules.

[0140] In this embodiment, the abnormal sound detection rule can be a rule configured in the abnormal sound detection device, a rule transmitted to the abnormal sound detection device from other devices, or rule information input by the user. The specific way the abnormal sound detection device obtains the abnormal sound detection rule can be determined according to the actual situation, and this embodiment does not limit it.

[0141] It should be noted that the abnormal sound detection rules include the start time of the abnormal sound at a constant frequency, as well as the frequency range of the abnormal sound at that constant frequency.

[0142] It should also be noted that the abnormal sound detection rules may include the termination time of abnormal sounds with constant frequency.

[0143] In this embodiment of the application, the abnormal sound detection device determines the abnormal sound detection result corresponding to the original audio based on constant frequency information and abnormal sound detection rules. The process includes: if the frequency corresponding to the constant frequency information is within the frequency range of the abnormal sound detection rules, and the start time corresponding to the constant frequency information is the same as the start time in the abnormal sound detection rules, the abnormal sound detection result is determined to be that the original audio is an abnormal sound; if the frequency corresponding to the constant frequency information is not within the frequency range of the abnormal sound detection rules, or the start time corresponding to the constant frequency information is different from the start time in the abnormal sound detection rules, the abnormal sound detection result is determined to be that the original audio is not an abnormal sound.

[0144] In this embodiment, if the abnormal sound detection rule further includes the termination time of the abnormal sound at a constant frequency, then if the frequency corresponding to the constant frequency information is within the frequency range of the abnormal sound detection rule, and the start time corresponding to the constant frequency information is the same as the start time in the abnormal sound detection rule, and the termination time corresponding to the constant frequency information is the same as the termination time in the abnormal sound detection rule, then the abnormal sound detection result is determined to be that the original audio is an abnormal sound. If the frequency corresponding to the constant frequency information is not within the frequency range of the abnormal sound detection rule, or the start time corresponding to the constant frequency information is different from the start time in the abnormal sound detection rule, or the termination time corresponding to the constant frequency information is different from the termination time in the abnormal sound detection rule, then the abnormal sound detection result is determined to be that the original audio is not an abnormal sound.

[0145] For example, regarding the detection of abnormal noises (HIIS) in air conditioners, these noises mainly occur in two frequency bands: 2500-3000 Hz and 4100-4300 Hz. Two sets of rules can be established: The first set targets noises in the 2500-3000 Hz range. The strategy (abnormal sound detection rule) could be: no frequency line within this range could be detected before the air conditioner is turned on, but a frequency line within this range would be present after the air conditioner is turned on. The second set targets abnormal noises in the 4100-4300 Hz range. The strategy (abnormal sound detection rule) could also be: no frequency line within this range could be detected before the air conditioner is turned on, but a frequency line within this range would be present after the air conditioner is turned on. If either of the two sets of rules is matched (i.e., the frequency corresponding to the constant frequency information is within the frequency range of the abnormal sound detection rule, and the start time corresponding to the constant frequency information is the same as the start time in the abnormal sound detection rule), it indicates that there is an abnormal noise from the air conditioner HIIS (i.e., the abnormal sound detection result is that the original audio is an abnormal sound).

[0146] In this embodiment, the abnormal sound detection device can first acquire the original audio collected during abnormal sound (such as the abnormal noise of an air conditioner HIIS) detection; then, it can segment the original audio according to a first segmentation step size (such as 5S) and a first window length (such as 10S) to obtain multiple audio segments; and perform short-time Fourier transform processing on the multiple audio segments to obtain a time-frequency matrix that describes the original audio in terms of time and frequency dimensions; then, it can acquire reference values ​​and the amplitude of each matrix parameter in the time-frequency matrix; adjust the amplitude based on the reference values ​​(such as adjusting the amplitude to decibels) to obtain a target matrix; perform edge detection (such as Canny edge detection) processing based on the matrix parameters in the target matrix to obtain a detection image; perform connectivity analysis filtering processing on the detection image to obtain a first image; and acquire hyperparameters for line detection (such as the hyperparameters for Hough line detection). The parameters include distance resolution in Hough space, angular resolution in Hough space, accumulator threshold, minimum line length, and maximum line segment distance, etc.; based on the hyperparameters, line detection (such as Hough line detection) is performed on the first image to obtain constant frequency information; finally, if the frequency corresponding to the constant frequency information is within the frequency range of the abnormal sound detection rules (such as 2500-3000 Hz or 4100-4300 Hz), and the start time corresponding to the constant frequency information is the same as the start time in the abnormal sound detection rules, the abnormal sound detection result is determined to be that the original audio is abnormal sound; if the frequency corresponding to the constant frequency information is not within the frequency range of the abnormal sound detection rules, or the start time corresponding to the constant frequency information is different from the start time in the abnormal sound detection rules, the abnormal sound detection result is determined to be that the original audio is not abnormal sound.

[0147] Understandably, by setting abnormal sound detection rules, the frequency and start time corresponding to the constant frequency information can be compared with the frequency range and start time in the abnormal sound detection rules to determine the abnormal sound detection result. It is not necessary to compare the decibel value obtained after performing FFT transformation on the frequency corresponding to the sound with the decibel threshold, thus avoiding false detection results in cases of inconsistent detection environments and improving the accuracy and stability of abnormal sound detection.

[0148] For example, such as Figure 4 The process involves: acquiring the original audio (16000Hz) collected during abnormal sound detection; segmenting the original audio according to the first segmentation step size and the first window length to obtain multiple audio segments; performing Short Time Fourier Transform (STFT) processing on the multiple audio segments to obtain a time-frequency matrix; acquiring reference values ​​and the amplitude of each matrix parameter in the time-frequency matrix; adjusting the amplitude based on the reference values ​​to obtain the target matrix; performing edge detection (Canny edge detection algorithm) on the target matrix to obtain a detection image; performing connectivity analysis filtering on the detection image to obtain a first image; acquiring hyperparameters for line detection; and performing line detection on the first image based on the hyperparameters to obtain a constant frequency signal. The information includes the duration and value of the constant frequency occurrence. If the frequency (frequency value) corresponding to the constant frequency information is within the frequency range of the abnormal sound detection rules (different judgment logic is formulated according to different business types), and the start time (starting position of the constant frequency occurrence duration) corresponding to the constant frequency information is the same as the start time in the abnormal sound detection rules, the abnormal sound detection result is determined to be that the original audio is an abnormal sound. If the frequency corresponding to the constant frequency information is not within the frequency range of the abnormal sound detection rules, or the start time corresponding to the constant frequency information is different from the start time in the abnormal sound detection rules, the abnormal sound detection result is determined to be that the original audio is not an abnormal sound.

[0149] For example, the result of using the abnormal sound detection method of this application to detect abnormal sounds in the acquired raw audio is as follows: Figure 5 As shown, multiple constant frequency information segments can be detected. By matching these segments with the abnormal sound detection rules, the abnormal sound detection results can be obtained.

[0150] Understandably, when an abnormal sound detection device acquires the original audio collected during abnormal sound detection, it determines the time-frequency matrix corresponding to the original audio and detects constant frequency information in the time-frequency matrix. Based on the constant frequency information and the abnormal sound detection rules, it determines the abnormal sound detection result corresponding to the original audio. It is not necessary to compare the decibel value obtained after performing FFT transformation on the frequency corresponding to the sound with the decibel threshold. This avoids false detection results due to inconsistent detection environments and improves the accuracy of abnormal sound detection.

[0151] Based on the same inventive concept as the above-mentioned abnormal sound detection method, this application provides an abnormal sound detection device 1, corresponding to an abnormal sound detection method; Figure 6 This is a schematic diagram of the structural composition of an abnormal sound detection device provided in an embodiment of this application. The abnormal sound detection device 1 may include:

[0152] Acquisition unit 11 is used to acquire the original audio collected during abnormal sound detection;

[0153] The determining unit 12 is used to determine the time-frequency matrix corresponding to the original audio; the time-frequency matrix is ​​a matrix describing the original audio in terms of time and frequency dimensions; and to determine the abnormal sound detection result corresponding to the original audio based on constant frequency information and abnormal sound detection rules.

[0154] The detection unit 13 is used to detect the constant frequency information in the time-frequency matrix.

[0155] In some embodiments of this application, the apparatus further includes a processing unit;

[0156] The detection unit 13 is used to perform constant frequency edge detection in the time-frequency matrix to obtain a detection image; and to perform line detection on the first image to obtain the constant frequency information.

[0157] The processing unit is used to perform connectivity analysis and filtering on the detected image to obtain a first image.

[0158] In some embodiments of this application, the device further includes an adjustment unit;

[0159] The acquisition unit 11 is used to acquire the reference value and the amplitude of each matrix parameter in the time-frequency matrix;

[0160] The adjustment unit is used to adjust the amplitude based on the reference value to obtain the target matrix;

[0161] The processing unit is used to perform edge detection processing based on the matrix parameters in the target matrix to obtain the detected image.

[0162] In some embodiments of this application, the acquisition unit 11 is used to acquire hyperparameters for line detection;

[0163] The detection unit 13 is used to perform line detection on the first image based on the hyperparameters to obtain the constant frequency information.

[0164] In some embodiments of this application, the processing unit is used to perform short-time Fourier transform processing on the original audio to obtain the time-frequency matrix.

[0165] In some embodiments of this application, the processing unit is configured to segment the original audio according to a first segmentation step size and a first window length to obtain multiple audio segments; and to perform short-time Fourier transform processing on the multiple audio segments to obtain the time-frequency matrix.

[0166] In some embodiments of this application, the determining unit 12 is configured to determine that the original audio is an abnormal sound when the frequency corresponding to the constant frequency information is within the frequency range of the abnormal sound detection rule and the start time corresponding to the constant frequency information is the same as the start time in the abnormal sound detection rule; and to determine that the original audio is not an abnormal sound when the frequency corresponding to the constant frequency information is not within the frequency range of the abnormal sound detection rule, or the start time corresponding to the constant frequency information is different from the start time in the abnormal sound detection rule.

[0167] It should be noted that, in practical applications, the acquisition unit 11, determination unit 12 and detection unit 13 mentioned above can be implemented by the processor 14 on the abnormal sound detection device, specifically by a CPU (Central Processing Unit), MPU (Microprocessor Unit), DSP (Digital Signal Processor) or Field Programmable Gate Array (FPGA), etc.; the data storage mentioned above can be implemented by the memory 15 on the abnormal sound detection device.

[0168] This application also provides an abnormal sound detection device, such as... Figure 7As shown, the abnormal sound detection device includes a processor 14, a memory 15, and a communication bus 16. The memory 15 communicates with the processor 14 through the communication bus 16. The memory 15 stores programs executable by the processor 14. When the program is executed, the abnormal sound detection method described above is executed by the processor 14.

[0169] In practical applications, the aforementioned memory 15 can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 14.

[0170] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of the abnormal sound detection device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the abnormal sound detection device to perform the abnormal sound detection method described in this application.

[0171] This application provides a computer-readable storage medium storing computer-executable instructions. When these computer-executable instructions are executed by a processor, they cause the processor to execute the abnormal sound detection method provided in this application. For example, ... Figure 1 The abnormal sound detection method is shown.

[0172] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEP ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0173] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0174] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0175] Understandably, when an abnormal sound detection device acquires the original audio collected during abnormal sound detection, it determines the time-frequency matrix corresponding to the original audio and detects constant frequency information in the time-frequency matrix. Based on the constant frequency information and the abnormal sound detection rules, it determines the abnormal sound detection result corresponding to the original audio. It is not necessary to compare the decibel value obtained after performing FFT transformation on the frequency corresponding to the sound with the decibel threshold. This avoids false detection results due to inconsistent detection environments and improves the accuracy of abnormal sound detection.

[0176] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0177] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0178] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0179] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0180] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for detecting abnormal sounds, characterized in that, The method includes: Acquire the raw audio collected during abnormal sound detection; Determine the time-frequency matrix corresponding to the original audio; the time-frequency matrix is ​​a matrix that describes the original audio in terms of time and frequency dimensions; Detect constant frequency information in the time-frequency matrix; Based on the constant frequency information and the abnormal sound detection rules, the abnormal sound detection result corresponding to the original audio is determined.

2. The method according to claim 1, characterized in that, The detection of constant frequency information in the time-frequency matrix includes: In the time-frequency matrix, constant frequency edge detection is performed to obtain a detection image; The detected image is subjected to connectivity analysis and filtering to obtain a first image; Line detection is performed on the first image to obtain the constant frequency information.

3. The method according to claim 2, characterized in that, The step of performing constant-frequency edge detection in the time-frequency matrix to obtain a detection image includes: Obtain the reference value and the amplitude of each matrix parameter in the time-frequency matrix; The amplitude is adjusted based on the reference value to obtain the target matrix; Edge detection is performed based on the matrix parameters in the target matrix to obtain the detected image.

4. The method according to any one of claims 2-3, characterized in that, The process of performing line detection on the first image to obtain the constant frequency information includes: Obtain the hyperparameters for line detection; Based on the hyperparameters, line detection is performed on the first image to obtain the constant frequency information.

5. The method according to any one of claims 1-4, characterized in that, Determining the time-frequency matrix corresponding to the original audio includes: The original audio is subjected to a short-time Fourier transform to obtain the time-frequency matrix.

6. The method according to claim 5, characterized in that, The step of performing a short-time Fourier transform on the original audio to obtain the time-frequency matrix includes: The original audio is segmented according to the first segmentation step size and the first window length to obtain multiple audio segments; The time-frequency matrix is ​​obtained by performing a short-time Fourier transform on the multiple audio segments.

7. The method according to any one of claims 1-6, characterized in that, The step of determining the abnormal sound detection result corresponding to the original audio based on the constant frequency information and the abnormal sound detection rules includes: If the frequency corresponding to the constant frequency information is within the frequency range of the abnormal sound detection rule, and the start time corresponding to the constant frequency information is the same as the start time in the abnormal sound detection rule, then the abnormal sound detection result is determined to be that the original audio is an abnormal sound. If the frequency corresponding to the constant frequency information is not within the frequency range of the abnormal sound detection rule, or if the start time corresponding to the constant frequency information is different from the start time in the abnormal sound detection rule, the abnormal sound detection result is determined to be that the original audio is not an abnormal sound.

8. An abnormal sound detection device, characterized in that, The device includes: The acquisition unit is used to acquire the raw audio collected during abnormal sound detection; A determining unit is used to determine the time-frequency matrix corresponding to the original audio; the time-frequency matrix is ​​a matrix describing the original audio in terms of time and frequency dimensions; and to determine the abnormal sound detection result corresponding to the original audio based on constant frequency information and abnormal sound detection rules. A detection unit is used to detect the constant frequency information in the time-frequency matrix.

9. An abnormal sound detection device, characterized in that, The abnormal sound detection device includes: Memory is used to store executable instructions for a computer; A processor, when executing computer-executable instructions stored in the memory, implements the abnormal sound detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed by a processor, they implement the method described in any one of claims 1 to 7.

11. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the method according to any one of claims 1 to 7.

Images