Method and device for identifying abnormal sound type, vehicle and storage medium

Abstract

The invention provides a method and device for identifying an abnormal sound type, a vehicle and a storage medium. According to the method, the spectrogram and the time domain graph of the sound signal to be detected are determined, and the amplitudes of N time points of the sound signal to be detected in the time domain graph are determined, so that the target time point when the sound signal to be detected is abnormal can be obtained. Furthermore, through the target time point and the amplitude of the target time point, image feature fusion is carried out on the time domain graph and the spectrogram, so that the time domain feature corresponding to the time point used for representing that the sound signal to be detected is abnormal in the time domain graph can be superposed on the spectrogram; the time domain features and the frequency domain features of the sound signals to be detected are better fused, and accurate fused sound features are obtained. Therefore, according to the method, the multi-angle features of the to-be-detected sound signal can be fused into an integral sound feature, a more accurate abnormal sound type of the to-be-detected sound signal is obtained by fusing the sound features, and the accuracy of performing abnormal type identification on the sound signal is improved.

Description

Technical Field

[0001] The present application relates to the field of sound testing, and more specifically, to a method, a device, a vehicle and a storage medium for identifying abnormal sound types in the field of sound testing. Background Art

[0002] With the development of vehicle technology and the improvement of people's living standards, more and more vehicles are entering people's lives. However, there are also more and more vehicle-related problems, including the problem of identifying abnormal types of vehicle-borne sounds in the vehicle.

[0003] In the related art, after determining the similarity between the actual sound signal in the vehicle and the abnormal sound signal, the abnormal sound type of the abnormal sound signal corresponding to the maximum similarity is determined as the abnormal sound type of the actual sound signal. The above process of identifying the abnormal type of vehicle-borne sound does not take into account the multi-angle characteristics of the vehicle-borne sound, resulting in the abnormal sound type determined when identifying the abnormal type of vehicle-borne sound is not accurate enough.

[0004] Therefore, there is an urgent need for a method for identifying abnormal sound types to improve the accuracy of abnormal type identification of sound signals. Summary of the invention

[0005] The present application provides a method, device, vehicle and storage medium for identifying abnormal sound types. The method can improve the accuracy of abnormal type identification of a sound signal to be tested.

[0006] In a first aspect, a method for identifying an abnormal sound type is provided, the method comprising: acquiring a sound signal to be tested, and determining a spectrogram and a time domain graph of the sound signal to be tested, each time point among N time points of the sound signal to be tested on the horizontal axis of the time domain graph corresponding to an amplitude of the sound signal to be tested on the vertical axis, N being a positive integer greater than 1; determining a target time point based on the amplitudes of the N time points, the target time point being a time point at which an abnormality occurs in the sound signal to be tested among the N time points; performing image feature fusion of the time domain graph and the spectrogram based on the target time point and the amplitude of the target time point to obtain a fused sound feature; and determining the abnormal sound type of the sound signal to be tested based on the fused sound feature.

[0007] In the above technical solution, the method determines the spectrogram and time domain graph of the sound signal to be tested, and determines the amplitude of N time points of the sound signal to be tested in the time domain graph, which can obtain the target time point at which the sound signal to be tested is abnormal. Further, the time domain graph and the spectrogram are fused with image features through the target time point and the amplitude of the target time point, which can superimpose the time domain features corresponding to the time point in the time domain graph used to characterize the abnormality of the sound signal to be tested on the spectrogram, and better fuse the time domain features and frequency domain features of the sound signal to be tested to obtain accurate fused sound features. Therefore, the method can fuse the multi-angle features of the sound signal to be tested into an overall sound feature (fused sound feature), obtain a more accurate abnormal sound type of the sound signal to be tested by fusing the sound features, and improve the accuracy of abnormal type identification of the sound signal.

[0008] In combination with the first aspect, in some possible implementations, the target time point is determined based on the amplitude of the N time points, including: for each adjacent first time point and second time point among the N time points, determining the absolute value of the difference between the amplitude of the second time point and the amplitude of the first time point; when the absolute value is greater than a preset threshold, determining the first time point and the second time point as the target time point.

[0009] In the above technical solution, taking the first time point and the second time point in any pair of adjacent time points as an example, by determining the absolute value of the difference between the amplitude of the second time point and the amplitude of the first time point, and comparing the absolute value with a preset threshold, it is possible to determine whether a large amplitude change occurs in the sound signal to be measured during the time period between the first time point and the second time point. When the absolute value is greater than the preset threshold, it indicates that a large amplitude change occurs in the sound signal to be measured. Under normal circumstances, the amplitude change of the sound signal is smooth, so the method determines the first time point and the second time point as the time points when the abnormality occurs in the sound signal to be measured. Therefore, the method can accurately determine the target time point.

[0010] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the method for determining the preset threshold includes: obtaining M abnormal sound signals from a sound library, each sound signal is marked with an abnormal time point at which the abnormality occurs, and M is a positive integer greater than 1; based on the abnormal time point, determining a target sound signal from the M abnormal sound signals, and the number of abnormal time points corresponding to the target sound signal is greater than a preset number; performing a wavelet transform on each target sound signal in the target sound signal to obtain a wavelet coefficient corresponding to each time point in P time points of each target sound signal, and P is a positive integer greater than 1; determining the preset threshold based on the average value of the difference between the wavelet coefficients corresponding to adjacent abnormal time points in the P time points of each target sound signal.

[0011] In the above technical solution, the vehicle controller obtains M abnormal sound signals from the sound library. This can avoid the influence of other types of sound signals (normal sound signals) on the determination of abnormal sound types, resulting in inaccurate determination of abnormal sound types. Furthermore, based on the abnormal time point, the target sound signal is determined from the M abnormal sound signals, which can expand the positive influence of the abnormal time point on the abnormal sound signal when determining the preset threshold, and obtain a more accurate preset threshold. Finally, each target sound signal in the target sound signal is subjected to wavelet transformation to obtain the wavelet coefficients corresponding to each time point in the P time points of each target sound signal, and further based on the average value of the difference between the wavelet coefficients corresponding to the adjacent abnormal time points in the P time points of each target sound signal, the preset threshold is determined. This can take into account the influence of the wavelet coefficients corresponding to the abnormal time points of each target sound signal on the preset threshold, and avoid extreme cases (the maximum difference corresponding to the wavelet coefficient and the minimum difference corresponding to the wavelet coefficient) affecting the determination process of the preset threshold. Therefore, the method can determine a more accurate preset threshold.

[0012] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, based on the target time point and the amplitude of the target time point, the time domain graph and the spectrogram are fused with image features to obtain a fused sound feature, including: grayscale processing of the spectrogram to obtain a grayscale spectrogram; based on the target time point and the amplitude of the target time point, the time domain graph and the grayscale spectrogram are fused with image features to obtain the fused sound feature.

[0013] In the above technical solution, before fusing the image features of the time domain graph and the spectrogram of the sound signal to be tested, the method grayscales the spectrogram to obtain a grayscale spectrogram. Compared with the spectrogram, the grayscale spectrogram can display the frequency components of the sound signal to be tested at different time points with a smaller amount of data. In this way, when the features of the sound signal to be tested in the frequency domain and the features in the time domain are subsequently fused, not only can the fusion steps be simplified and the amount of calculation be reduced, but also more accurate fused sound features can be determined.

[0014] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the amplitude on the vertical axis of the time domain graph is a wavelet coefficient, and each time point of the sound signal to be measured on the horizontal axis of the grayscale spectrogram corresponds to an energy grayscale value of the sound signal to be measured, and based on the target time point and the amplitude of the target time point, the time domain graph and the grayscale spectrogram are subjected to image feature fusion, including: determining a candidate time point that matches the target time point among the N time points on the horizontal axis of the grayscale spectrogram; obtaining a current energy grayscale value corresponding to the candidate time point on the horizontal axis of the grayscale spectrogram; and adjusting the current energy grayscale value based on the wavelet coefficient corresponding to the target time point to perform image feature fusion of the time domain graph and the grayscale spectrogram.

[0015] In the above technical solution, the method determines a candidate time point that matches the target time point among the N time points on the horizontal axis of the grayscale spectrogram. This can accurately determine multiple candidate time points for image feature fusion on the grayscale spectrogram. Furthermore, by obtaining the current energy grayscale value corresponding to the candidate time point on the horizontal axis of the grayscale spectrogram, the basic change amount when performing image feature fusion can be determined to avoid the situation where the corresponding current energy grayscale value cannot be found and the image feature fusion cannot be performed. Furthermore, the method adjusts the current energy grayscale value based on the wavelet coefficient corresponding to the target time point, which can superimpose the energy (amplitude) feature used to characterize the sound signal to be tested in the time domain diagram on the speech energy in the grayscale spectrogram, so that the fused sound feature can more accurately characterize the speech energy of the sound signal to be tested. Therefore, the method can accurately determine the abnormal sound type of the sound signal to be tested by fusing the sound feature.

[0016] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the current energy grayscale value is adjusted based on the wavelet coefficient corresponding to the target time point, including: normalizing the wavelet coefficient corresponding to the target time point to obtain the normalized wavelet coefficient corresponding to the target time point; determining the product of the current energy grayscale value and the normalized wavelet coefficient corresponding to the target time point as the adjustment amount of the grayscale value; determining the sum of the current energy grayscale value and the adjustment amount as the target energy grayscale value; and adjusting the current energy grayscale value corresponding to the candidate time point to the target energy grayscale value.

[0017] In the above technical solution, the method normalizes the wavelet coefficients corresponding to the target time point, and can obtain the normalized wavelet coefficients corresponding to the target time point, ensuring that the value range of the normalized wavelet coefficients is [0, 1]. In this way, the method can increase the current energy gray value based on the normalized wavelet coefficients, change the level of each point (gray point) in the grayscale spectrogram (the degree to which the gray level is aggravated), and more accurately fuse the image features of the time domain graph and the spectrogram of the sound signal to be tested to obtain the fused sound feature.

[0018] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, based on the fused sound feature, determining the abnormal sound type of the sound signal to be tested includes: inputting the fused sound feature into a sound classification model to determine the abnormal sound type of the sound signal to be tested, the sound classification model is used to determine the abnormal type of the sound signal, and the sound classification model is trained by multiple abnormal sound signals and sample abnormal types corresponding to each abnormal sound signal.

[0019] In the above technical solution, the method inputs the fused sound feature into the sound classification model, and the sound classification model processes the fused sound feature to obtain the abnormal sound type of the sound signal to be tested. Since the sound classification model is trained based on a large number of data sets of abnormal sound signals, the sound classification model can improve the accuracy of abnormal type identification of the sound signal to be tested and improve the rate of determining the abnormal sound type of the sound signal to be tested.

[0020] In a second aspect, a device for identifying abnormal sound types is provided, the device comprising: an acquisition module, used to acquire a sound signal to be tested, and determine a spectrogram and a time domain diagram of the sound signal to be tested, each time point of N time points of the sound signal to be tested on the horizontal axis of the time domain diagram corresponds to an amplitude of the sound signal to be tested on the vertical axis, and N is a positive integer greater than 1; a determination module, used to: determine a target time point based on the amplitudes of the N time points, the target time point being a time point at which an abnormality occurs in the sound signal to be tested among the N time points; based on the target time point and the amplitude of the target time point, perform image feature fusion of the time domain diagram and the spectrogram to obtain a fused sound feature; and determine the abnormal sound type of the sound signal to be tested based on the fused sound feature.

[0021] In combination with the second aspect, in some possible implementations, the determination module is specifically used to: for each adjacent first time point and second time point among the N time points, determine the absolute value of the difference between the amplitude of the second time point and the amplitude of the first time point; when the absolute value is greater than a preset threshold, determine the first time point and the second time point as the target time point.

[0022] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the device also includes: an acquisition module, used to obtain M abnormal sound signals from a sound library, each sound signal is marked with an abnormal time point where the abnormality occurs, and M is a positive integer greater than 1; the determination module is specifically used to: determine a target sound signal from the M abnormal sound signals based on the abnormal time point, and the number of abnormal time points corresponding to the target sound signal is greater than a preset number; perform a wavelet transform on each target sound signal in the target sound signal to obtain a wavelet coefficient corresponding to each time point in the P time points of each target sound signal, and P is a positive integer greater than 1; determine the preset threshold based on the average value of the difference between the wavelet coefficients corresponding to adjacent abnormal time points in the P time points of each target sound signal.

[0023] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the determination module is specifically used to grayscale the spectrogram to obtain a grayscale spectrogram; the device also includes: a feature fusion module, which is used to fuse the image features of the time domain image and the grayscale spectrogram based on the target time point and the amplitude of the target time point to obtain the fused sound feature.

[0024] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the amplitude on the vertical axis of the time domain graph is the wavelet coefficient, and each time point of the sound signal to be measured on the horizontal axis of the grayscale spectrogram corresponds to an energy grayscale value of the sound signal to be measured. The determination module is also specifically used to determine the candidate time point that matches the target time point among the N time points on the horizontal axis of the grayscale spectrogram; the acquisition module is specifically used to obtain the current energy grayscale value corresponding to the candidate time point on the horizontal axis of the grayscale spectrogram; the feature fusion module is specifically used to adjust the current energy grayscale value based on the wavelet coefficient corresponding to the target time point, so as to perform image feature fusion of the time domain graph and the grayscale spectrogram.

[0025] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the determination module is specifically used to: normalize the wavelet coefficient corresponding to the target time point to obtain the normalized wavelet coefficient corresponding to the target time point; determine the product of the current energy grayscale value and the normalized wavelet coefficient corresponding to the target time point as the adjustment amount of the grayscale value; determine the sum of the current energy grayscale value and the adjustment amount as the target energy grayscale value; the feature fusion module is specifically used to adjust the current energy grayscale value corresponding to the candidate time point to the target energy grayscale value.

[0026] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the determination module is specifically used to input the fused sound feature into a sound classification model to determine the abnormal sound type of the sound signal. The sound classification model is used to determine the abnormal type of the sound signal to be tested. The sound classification model is obtained by training multiple abnormal sound signals and sample abnormal types corresponding to each abnormal sound signal.

[0027] In a third aspect, a vehicle is provided, comprising a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the vehicle executes the method in the first aspect or any possible implementation of the first aspect.

[0028] In a fourth aspect, a computer-readable storage medium is provided, which stores an executable program code. When the executable program code runs on a computer, the computer executes the method in the above-mentioned first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a schematic diagram of a scenario of using a vehicle provided in an embodiment of the present application;

[0030] Figure 2 is a schematic flow chart of a method for identifying abnormal sound types provided in an embodiment of the present application;

[0031] Figure 3 is a schematic diagram of determining N time points of a sound signal to be tested provided by an embodiment of the present application;

[0032] Figure 4 is a block diagram of determining an abnormal sound type of a sound signal to be tested provided by an embodiment of the present application;

[0033] Figure 5 It is a structural schematic diagram of a device for identifying abnormal sound types provided in an embodiment of the present application;

[0034] Figure 6 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0036] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0037] Figure 1 It is a schematic diagram of a scenario of using a vehicle provided in an embodiment of the present application.

[0038] For example, Figure 1 As shown, the user in vehicle A can listen to music through the vehicle system to meet the user's entertainment needs. However, when the user is listening to music, the music signal may be abnormal. In the related art, after determining the similarity between the actual music signal and the abnormal music signal, the abnormal sound type of the abnormal music signal corresponding to the maximum similarity is determined as the abnormal sound type of the actual music signal, so as to inform the user of the abnormal situation of the abnormal sound type of the music signal.

[0039] The multi-angle characteristics of the music signal are not taken into account in the above process of identifying the abnormal type of the music signal, resulting in the abnormal sound type determined when identifying the abnormal type of the music signal being inaccurate, thereby misleading the user to correct the abnormality of the above music signal.

[0040] The following will take the above-mentioned abnormal type identification of the sound signal in the vehicle as an example for description. However, it should be understood that the present application is not limited to this. In fact, the present application is also applicable to the abnormal type identification of other types of sound signals, for example, pathological sound signals, specifically including heart murmur signals and lung rales signals, and for example, natural disaster sound signals, specifically including sound signals corresponding to tornadoes, sound signals corresponding to earthquakes, and sound signals corresponding to floods.

[0041] In order to solve the above problems, this application proposes a method for identifying abnormal sound types to improve the accuracy of abnormal type identification of sound signals. For details, please refer to the following Figure 2 The implementation steps are shown.

[0042] Figure 2It is a schematic flowchart of a method for identifying abnormal sound types provided in an embodiment of the present application.

[0043] It should be understood that the method for identifying abnormal sound types provided in the embodiment of the present application can be applied to Figure 1 The vehicle shown (eg, vehicle A). Specifically, the method for identifying the type of abnormal sound can be applied to a vehicle controller in the vehicle.

[0044] For example, Figure 2 As shown, the method 200 includes the following steps 201 to 204.

[0045] In step 201, the vehicle controller collects the sound signal to be tested and determines the spectrogram and time domain diagram of the sound signal to be tested. Each of the N time points of the sound signal to be tested on the horizontal axis of the time domain diagram corresponds to an amplitude of the sound signal to be tested on the vertical axis, and N is a positive integer greater than 1.

[0046] It should be understood that, in some embodiments, the "sound signal to be tested" in the above step 201 is a vehicle-mounted sound signal to be tested.

[0047] It should also be understood that the "spectrogram of the sound signal to be tested" in the above step 201 is used to reflect the sound characteristics of the sound signal to be tested in the frequency domain, and the spectrogram can display the frequency components of the sound signal to be tested at different time points. The horizontal axis of the spectrogram is the time point of the sound signal to be tested, and the vertical axis is the frequency of the sound signal to be tested. The color depth of each point on the spectrogram is used to characterize the speech energy of the sound signal to be tested corresponding to {time point, frequency}. Specifically, the darker the color of any point, the greater the speech energy of the sound signal to be tested represented at that point.

[0048] It should also be understood that the "time domain diagram of the sound signal to be tested" in the above step 201 is used to reflect the sound characteristics of the sound signal to be tested in the time domain, and the time domain diagram can show the amplitude changes of the sound signal to be tested at different time points. The horizontal axis of the time domain diagram is the time point of the sound signal to be tested, and the vertical axis is the amplitude of the sound signal to be tested.

[0049] In some embodiments, an output harness of a voice playback device in a vehicle is connected to a sound card, the sound card is connected to a sound testing device, and a vehicle controller collects a sound signal to be tested, including: upon receiving a trigger operation to record the sound signal to be tested output by the output harness on the sound testing device, or when the output harness outputs the sound signal to be tested, the vehicle controller collects the sound signal to be tested from the output harness through the sound card and the sound testing device.

[0050] It should be understood that the "sound card" in the above scheme refers to a recording sound card, which can convert the analog sound signal to be tested (the analog sound signal corresponding to the sound signal to be tested) into a digital sound signal (the digital sound signal to be tested), and then restore the digital sound signal to an analog sound signal, thereby ensuring that the vehicle controller collects high-definition and high-quality sound signals to be tested. In addition, the vehicle controller collects the sound signal to be tested from the output harness through the sound card and the sound test equipment based on the Pyaudio library for processing audio.

[0051] In some embodiments, the sound card is a Roland sound card.

[0052] In some embodiments, the voice playback device is a speaker.

[0053] In some embodiments, the sound testing device is a vehicle-mounted display screen.

[0054] In some embodiments, the vehicle controller determines the spectrogram of the sound signal to be tested, including: the vehicle controller converts the analog sound signal corresponding to the sound signal to be tested into a digital sound signal; the vehicle controller performs frame processing on the digital sound signal to obtain Q digital sound segments of the same length, where Q is a positive integer greater than 0; the vehicle controller performs windowing processing on each digital sound segment to obtain Q processed digital sound segments; the vehicle controller performs Fourier transform on each digital sound segment to obtain the frequency spectrum of each digital sound segment; the vehicle controller performs modulo processing on the frequency spectrum of each digital sound segment to obtain the amplitude spectrum of each digital sound segment, which is used to indicate the amplitude of the sound signal to be tested at different frequencies; the vehicle controller generates the spectrogram based on the amplitude spectrum of each digital sound segment in the Q digital sound segments.

[0055] It should be understood that the "Q digital sound segments of the same length" in the above solution can determine N time points of the sound signal to be tested, and the relationship between Q and N is: N = Q + 1. In addition, the vehicle controller can determine the spectrogram of the sound signal to be tested with the help of the Librosa library for audio and music analysis.

[0056] It should also be understood that the "amplitude spectrum" in the above scheme can be regarded as speech energy.

[0057] In some embodiments, the vehicle controller generates the spectrogram based on the amplitude spectra of each digital sound segment in Q digital sound segments of the same length, including: the vehicle controller takes any digital sound segment in the Q digital sound segments of the same length as the target digital sound segment, and determines the first starting time point and the first ending time point of the target digital sound segment; the vehicle controller determines that the N time points on the horizontal axis of the spectrogram include the first starting time point and the first ending time point, determines that the multiple frequencies on the vertical axis of the spectrogram include the first frequency of the target digital sound segment corresponding to the first starting time point and the second frequency of the target digital sound segment corresponding to the first ending time point, and determines that the speech energy corresponding to the first starting time point and the first frequency is the amplitude of the target digital sound segment at the first frequency, and the speech energy corresponding to the first ending time point and the second frequency is the amplitude of the target digital sound segment at the second frequency.

[0058] In some embodiments, the vehicle controller determines a time domain diagram of the sound signal to be tested, including: the vehicle controller takes any one of Q digital sound segments of the same length as a target digital sound segment, and determines a first starting time point and a first ending time point of the target digital sound segment; the vehicle controller determines that the N time points on the horizontal axis of the time domain diagram include the first starting time point and the first ending time point, and determines that multiple amplitudes on the vertical axis of the time domain diagram include a first amplitude of the target digital sound segment corresponding to the first starting time point and a second amplitude of the target digital sound segment corresponding to the first ending time point.

[0059] Figure 3 It is a schematic diagram of determining N time points of a sound signal to be tested provided by an embodiment of the present application.

[0060] For example, Figure 3 As shown, the vehicle controller determines the frame length, for example, 300ms. Based on the frame length, the vehicle controller performs frame processing on the digital sound signal corresponding to the sound signal to be tested of 3s (3000ms), and obtains 10 (3000 / 300) digital sound clips of the same length. The vehicle controller determines that N is 11 (10+1), that is, 11 time points, through 10 digital sound clips of the same length, and the times of the 11 time points are: 0ms, 300ms, 600ms, 900ms, 1200ms, 1500ms, 1800ms, 2100ms, 2400ms, 2700ms and 3000ms.

[0061] Step 202: The vehicle controller determines a target time point based on the amplitudes of the N time points. The target time point is a time point at which the sound signal to be measured becomes abnormal among the N time points.

[0062] In one possible implementation, the vehicle controller in step 202 determines the target time point based on the amplitudes of the N time points, including: for each adjacent first time point and second time point among the N time points, the vehicle controller determines the absolute value of the difference between the amplitude of the second time point and the amplitude of the first time point; when the absolute value is greater than a preset threshold, the vehicle controller determines the first time point and the second time point as the target time point.

[0063] It should be understood that the "first time point and the second time point of each adjacent time point in the N time points" in the above scheme refers to each pair of adjacent time points in the Q pairs of adjacent time points in the N time points, and each pair of adjacent time points is the first time point and the second time point. Generally, if the difference between the amplitudes of adjacent time points is large, it indicates that the sound signal of the period corresponding to the adjacent time points is abnormal.

[0064] It should also be understood that in the above scheme, any pair of adjacent time points among Q pairs of adjacent time points (including the first time point and the second time point) is taken as an example to describe the process of determining the target time point.

[0065] It should also be understood that the "preset threshold" in the above solution is closely related to the sound characteristics of the abnormal sound signal. In some embodiments, the preset threshold is also related to the personal judgment of the sound tester.

[0066] In the above technical solution, taking the first time point and the second time point in any pair of adjacent time points as an example, by determining the absolute value of the difference between the amplitude of the second time point and the amplitude of the first time point, and comparing the absolute value with a preset threshold, it is possible to determine whether a large amplitude change occurs in the sound signal to be measured during the time period between the first time point and the second time point. When the absolute value is greater than the preset threshold, it indicates that a large amplitude change occurs in the sound signal to be measured. Under normal circumstances, the amplitude change of the sound signal is smooth, so the method determines the first time point and the second time point as the time points when the abnormality occurs in the sound signal to be measured. Therefore, the method can accurately determine the target time point.

[0067] In one possible implementation, the method for determining the preset threshold includes: the vehicle controller obtains M abnormal sound signals from the sound library, each sound signal is marked with an abnormal time point when the abnormality occurs, and M is a positive integer greater than 1; the vehicle controller determines a target sound signal from the M abnormal sound signals based on the abnormal time point, and the number of abnormal time points corresponding to the target sound signal is greater than a preset number; the vehicle controller performs a wavelet transform on each target sound signal in the target sound signal to obtain a wavelet coefficient corresponding to each time point in P time points of each target sound signal, and P is a positive integer greater than 1; the vehicle controller determines the preset threshold based on the average value of the difference between the wavelet coefficients corresponding to adjacent abnormal time points in the P time points of each target sound signal.

[0068] It should be understood that the "sound library" in the above scheme is a sound signal obtained from a variety of sound signal sources, including abnormal sound signals and normal sound signals. The abnormal sound signals include abnormal vehicle-mounted sound signals, abnormal pathological sound signals, and abnormal natural disaster sound signals. In some embodiments, the abnormal vehicle-mounted sound signal includes a signal of sound freeze, a signal of sound mixing, and an electric current sound signal output by a voice playback device. In some embodiments, the abnormal pathological sound signal includes a heart murmur signal and a lung rales signal. In some embodiments, the abnormal natural disaster sound signal includes a sound signal corresponding to a tornado, a sound signal corresponding to an earthquake, and a sound signal corresponding to a flood.

[0069] It should also be understood that the "abnormal time points at which the abnormalities occur in each sound signal" in the above scheme appear in pairs, and the adjacent abnormal time points of each sound signal at least include the amplitude of the sound signal corresponding to the fourth time point among the adjacent abnormal time points being much larger than the amplitude of the sound signal corresponding to the third time point, and also include the speech energy of the sound signal corresponding to the fourth time point being much larger than the speech energy of the sound signal corresponding to the third time point.

[0070] It should also be understood that in the above scheme, each target sound signal can be subjected to wavelet transform by a target wavelet function. In some embodiments, the target wavelet function includes any one of a Haar wavelet function, a Daubechies wavelet function, a Morlet wavelet function, and a Mexican Hat wavelet function. In addition, the value of the wavelet coefficient corresponding to each time point is an arbitrary real number.

[0071] In the above technical solution, the vehicle controller obtains M abnormal sound signals from the sound library. This can avoid the influence of other types of sound signals (normal sound signals) on the determination of abnormal sound types, resulting in inaccurate determination of abnormal sound types. Furthermore, based on the abnormal time point, the target sound signal is determined from the M abnormal sound signals, which can expand the positive influence of the abnormal time point on the abnormal sound signal when determining the preset threshold, and obtain a more accurate preset threshold. Finally, each target sound signal in the target sound signal is subjected to wavelet transformation to obtain the wavelet coefficients corresponding to each time point in the P time points of each target sound signal, and further based on the average value of the difference between the wavelet coefficients corresponding to the adjacent abnormal time points in the P time points of each target sound signal, the preset threshold is determined. This can take into account the influence of the wavelet coefficients corresponding to the abnormal time points of each target sound signal on the preset threshold, and avoid extreme cases (the maximum difference corresponding to the wavelet coefficient and the minimum difference corresponding to the wavelet coefficient) affecting the determination process of the preset threshold. Therefore, the method can determine a more accurate preset threshold.

[0072] In some embodiments, the preset number is 7.

[0073] Step 203: The vehicle controller fuses the image features of the time domain graph and the spectrogram based on the target time point and the amplitude at the target time point to obtain a fused sound feature.

[0074] It should be understood that the horizontal axes of the time domain graph and the spectrogram are both N time points of the sound signal to be tested, the vertical axis of the spectrogram is the frequency of the sound signal to be tested, the color depth of each point on the spectrogram is used to characterize the speech energy of the sound signal to be tested corresponding to {time point, frequency}, the vertical axis of the time domain graph is the amplitude of the sound signal to be tested, and the speech energy of the sound signal to be tested is proportional to the square of the amplitude. Therefore, the process of "fusing the time domain graph and the spectrogram for image features" in the above step 203 can be implemented from the amplitude of the sound signal to be tested in the time domain graph and the speech energy of the sound signal to be tested in the spectrogram (these two angles) to fuse the multi-angle features of the sound signal to be tested into an overall sound feature (fused sound feature).

[0075] In a possible implementation, step 203 includes: the vehicle controller grayscales the spectrogram to obtain a grayscale spectrogram; the vehicle controller performs image feature fusion on the time domain graph and the grayscale spectrogram based on the target time point and the amplitude at the target time point to obtain the fused sound feature.

[0076] It should be understood that the "grayscale processing of the spectrogram" in the above scheme refers to converting the speech energy of the sound signal to be tested represented by the color in the spectrogram into the speech energy of the sound signal to be tested represented by the gray value (which can be called the energy gray value). After the spectrogram is grayscaled, the color in the spectrogram is converted into different levels of gray in the grayscale spectrogram. In addition, the vehicle controller can grayscale the spectrogram with the help of the OpenCV library for image processing to obtain a grayscale spectrogram.

[0077] In the above technical solution, before fusing the image features of the time domain graph and the spectrogram of the sound signal to be tested, the method grayscales the spectrogram to obtain a grayscale spectrogram. Compared with the spectrogram, the grayscale spectrogram can display the frequency components of the sound signal to be tested at different time points with a smaller amount of data. In this way, when the features of the sound signal to be tested in the frequency domain and the features in the time domain are subsequently fused, not only can the fusion steps be simplified and the amount of calculation be reduced, but also more accurate fused sound features can be determined.

[0078] In one possible implementation, the amplitude on the vertical axis of the time domain graph is the wavelet coefficient, and each time point of the sound signal to be measured on the horizontal axis of the grayscale spectrogram corresponds to an energy grayscale value of the sound signal to be measured. The vehicle controller fuses the time domain graph and the grayscale spectrogram based on the target time point and the amplitude of the target time point, including: the vehicle controller determines a candidate time point that matches the target time point among the N time points on the horizontal axis of the grayscale spectrogram; the vehicle controller obtains the current energy grayscale value corresponding to the candidate time point on the horizontal axis of the grayscale spectrogram; the vehicle controller adjusts the current energy grayscale value based on the wavelet coefficient corresponding to the target time point to fuse the time domain graph and the grayscale spectrogram.

[0079] It should be understood that, generally, the "target time point" in the above scheme includes R time points, R is a positive integer greater than 1 and R is less than or equal to N, and the "candidate time point matching the target time point" refers to a candidate time point that is identical to each of the R time points. In other words, the candidate time points determined from the N time points on the horizontal axis of the grayscale spectrogram include R, and the R candidate time points correspond to the R time points one by one.

[0080] It should also be understood that there are multiple "current energy grayscale values ​​corresponding to the candidate time point on the horizontal axis of the grayscale spectrogram" in the above scheme, because the frequency of the sound signal to be tested on the vertical axis of the grayscale spectrogram is not considered. However, the multiple current energy grayscale values ​​are usually different.

[0081] It should also be understood that after the above scheme "adjusts the current energy grayscale value based on the wavelet coefficient corresponding to the target time point to fuse the time domain image and the grayscale spectrogram for image features", the gray level (gray point) of each point on the grayscale spectrogram is deepened and the gray degree is intensified. In other words, the fused sound feature can be regarded as a grayscale spectrogram with the level of each gray point deepened.

[0082] In the above technical solution, the method determines a candidate time point that matches the target time point among the N time points on the horizontal axis of the grayscale spectrogram. This can accurately determine multiple candidate time points for image feature fusion on the grayscale spectrogram. Furthermore, by obtaining the current energy grayscale value corresponding to the candidate time point on the horizontal axis of the grayscale spectrogram, the basic change amount when performing image feature fusion can be determined to avoid the situation where the corresponding current energy grayscale value cannot be found and the image feature fusion cannot be performed. Furthermore, the method adjusts the current energy grayscale value based on the wavelet coefficient corresponding to the target time point, which can superimpose the energy (amplitude) feature used to characterize the sound signal to be tested in the time domain diagram on the speech energy in the grayscale spectrogram, so that the fused sound feature can more accurately characterize the speech energy of the sound signal to be tested. Therefore, the method can accurately determine the abnormal sound type of the sound signal to be tested by fusing the sound feature.

[0083] In some embodiments, the amplitude on the vertical axis of the time domain graph is a method for determining the wavelet coefficient, including: the vehicle controller performs a wavelet transform on the sound signal to be measured through the target wavelet function to obtain the wavelet coefficient; the vehicle controller takes any one of the Q digital sound segments of the same length as the target digital sound segment, and determines the first starting time point and the first ending time point of the target digital sound segment; the vehicle controller determines that the N time points on the horizontal axis of the time domain graph include the first starting time point and the first ending time point, and determines that the multiple amplitudes on the vertical axis of the time domain graph include the first wavelet coefficient of the target digital sound segment corresponding to the first starting time point, and the second wavelet coefficient of the target digital sound segment corresponding to the first ending time point.

[0084] In one possible implementation, the vehicle controller adjusts the current energy grayscale value based on the wavelet coefficient corresponding to the target time point, including: the vehicle controller normalizes the wavelet coefficient corresponding to the target time point to obtain the normalized wavelet coefficient corresponding to the target time point; the vehicle controller determines the product of the current energy grayscale value and the normalized wavelet coefficient corresponding to the target time point as the adjustment amount of the grayscale value; the vehicle controller determines the sum of the current energy grayscale value and the adjustment amount as the target energy grayscale value; the vehicle controller adjusts the current energy grayscale value corresponding to the candidate time point to the target energy grayscale value.

[0085] It should be understood that after "normalizing the wavelet coefficients corresponding to the target time point" in the above scheme, the value range of the normalized wavelet coefficients obtained is [0, 1], and the normalized wavelet coefficients can be regarded as the amplitude value of increasing the current energy grayscale value. For example, when the normalized wavelet coefficient is 0.5, the amplitude of increasing the current energy grayscale value by 50%, that is, the adjusted energy grayscale value is: current energy grayscale value + current energy grayscale value * 50%.

[0086] It should also be understood that after “adjusting the current energy grayscale value” in the above solution, the level of each gray point on the grayscale spectrogram is deepened and the gray degree is intensified.

[0087] In the above technical solution, the method normalizes the wavelet coefficients corresponding to the target time point, and can obtain the normalized wavelet coefficients corresponding to the target time point, ensuring that the value range of the normalized wavelet coefficients is [0, 1]. In this way, the method can increase the current energy gray value based on the normalized wavelet coefficients, change the level of each point (gray point) in the grayscale spectrogram (the degree to which the gray level is aggravated), and more accurately fuse the image features of the time domain graph and the spectrogram of the sound signal to be tested to obtain the fused sound feature.

[0088] Step 204: The vehicle controller determines the abnormal sound type of the sound signal to be tested based on the fused sound feature.

[0089] It should be understood that the "abnormal sound type" in the above step 204 refers to the classification of various abnormal phenomena when the sound signal to be tested has sound characteristics (abnormal phenomena) that do not meet the expected ones.

[0090] In one possible implementation, step 204 includes: the vehicle controller inputs the fused sound feature into a sound classification model to determine the abnormal sound type of the sound signal to be tested. The sound classification model is used to determine the abnormal type of the sound signal to be tested. The sound classification model is trained by multiple abnormal sound signals and sample abnormal types corresponding to each abnormal sound signal.

[0091] It should be understood that the "sound classification model" in the above solution is pre-trained based on a convolutional neural network through multiple abnormal sound signals and the abnormality type corresponding to each abnormal sound signal. The sound classification model can accurately and quickly determine the abnormal type of the sound signal to be tested.

[0092] In the above technical solution, the method inputs the fused sound feature into the sound classification model, and the sound classification model processes the fused sound feature to obtain the abnormal sound type of the sound signal to be tested. Since the sound classification model is trained based on a large number of data sets of abnormal sound signals, the sound classification model can improve the accuracy of abnormal type identification of the sound signal to be tested and improve the rate of determining the abnormal sound type of the sound signal to be tested.

[0093] In some embodiments, the method for determining the sound classification model includes: the vehicle controller constructs an initial sound classification model based on a convolutional neural network; the vehicle controller determines the spectrogram and time domain graph of each abnormal sound signal among the multiple abnormal sound signals; for any abnormal sound signal, the vehicle controller performs image feature fusion on the time domain graph of the abnormal sound signal and the corresponding spectrogram to obtain a sample fusion sound feature; the vehicle controller inputs the sample fusion sound feature corresponding to the first abnormal sound signal among the multiple abnormal sound signals into the initial sound classification model, and the initial sound classification model determines the abnormal type of the first abnormal sound signal, determines a first loss value between the abnormal type of the first abnormal sound signal and the corresponding sample abnormal type, and when the first loss value is greater than a preset loss value, adjusts the parameters in the initial sound classification model to obtain a first sound classification model; the vehicle controller inputs the sample fusion sound feature corresponding to the first abnormal sound signal among the multiple abnormal sound signals into the initial sound classification model, and determines the first abnormal type of the first abnormal sound signal. The first loss value is greater than a preset loss value. The parameters in the initial sound classification model are adjusted to obtain a first sound classification model. The sample fusion sound features corresponding to the abnormal sound signal are input into the first sound classification model, and the first sound classification model determines the abnormal type of the second abnormal sound signal, and determines the second loss value between the abnormal type of the second abnormal sound signal and the corresponding sample abnormal type. When the second loss value is greater than the preset loss value, the parameters in the first sound classification model are adjusted to obtain the second sound classification model; and so on, until the vehicle controller inputs the sample fusion sound features corresponding to the Kth abnormal sound signal among the multiple abnormal sound signals into the K-1th sound classification model, and the K-1th sound classification model determines the abnormal type of the Kth abnormal sound signal, and determines the Kth loss value between the abnormal type of the Kth abnormal sound signal and the corresponding sample abnormal type. When the Kth loss value is less than or equal to the preset loss value, the K-1th sound classification model is determined as the sound classification model, and K is a positive integer greater than 3.

[0094] Figure 4 It is a block diagram for determining the abnormal sound type of a sound signal to be tested provided by an embodiment of the present application.

[0095] For example, Figure 4As shown, the vehicle controller collects the sound signal to be tested, determines the time domain graph and spectrogram of the sound signal to be tested, and grayscales the spectrogram to obtain a grayscale spectrogram. The vehicle controller fuses the image features of the time domain graph and the grayscale spectrogram to obtain a fused sound feature. The vehicle controller inputs the fused sound feature into the sound classification model, and the sound classification model determines the abnormal sound type of the sound signal to be tested.

[0096] Figure 5 It is a structural schematic diagram of a device for identifying abnormal sound types provided in an embodiment of the present application.

[0097] For example, Figure 5 As shown, the device 500 includes:

[0098] The acquisition module 501 is used to acquire the sound signal to be tested, and determine the spectrogram and time domain diagram of the sound signal to be tested, each time point of N time points of the sound signal to be tested on the horizontal axis of the time domain diagram corresponds to an amplitude of the sound signal to be tested on the vertical axis, and N is a positive integer greater than 1;

[0099] The determination module 502 is used to:

[0100] Based on the amplitudes of the N time points, determining a target time point, the target time point being a time point at which the sound signal to be tested is abnormal among the N time points;

[0101] Based on the target time point and the amplitude at the target time point, the time domain image and the spectrogram are fused with image features to obtain a fused sound feature;

[0102] Based on the fused sound feature, the abnormal sound type of the sound signal to be tested is determined.

[0103] Optionally, the determination module 502 is specifically used to: for each adjacent first time point and second time point among the N time points, determine the absolute value of the difference between the amplitude of the second time point and the amplitude of the first time point; when the absolute value is greater than a preset threshold, determine the first time point and the second time point as the target time point.

[0104] Optionally, the device 500 also includes: an acquisition module, used to obtain M abnormal sound signals from a sound library, each sound signal is marked with an abnormal time point when the abnormality occurs, and M is a positive integer greater than 1; the determination module 502 is specifically used to: determine a target sound signal from the M abnormal sound signals based on the abnormal time point, and the number of abnormal time points corresponding to the target sound signal is greater than a preset number; perform a wavelet transform on each target sound signal in the target sound signal to obtain a wavelet coefficient corresponding to each time point in the P time points of each target sound signal, and P is a positive integer greater than 1; determine the preset threshold based on the average value of the difference between the wavelet coefficients corresponding to adjacent abnormal time points in the P time points of each target sound signal.

[0105] Optionally, the determination module 502 is further used to grayscale the spectrogram to obtain a grayscale spectrogram; the device 500 also includes: a feature fusion module, which is used to fuse the image features of the time domain image and the grayscale spectrogram based on the target time point and the amplitude of the target time point to obtain the fused sound feature.

[0106] Optionally, the amplitude on the vertical axis of the time domain graph is the wavelet coefficient, and each time point of the sound signal to be measured on the horizontal axis of the grayscale spectrogram corresponds to an energy grayscale value of the sound signal to be measured. The determination module 502 is also specifically used to determine a candidate time point that matches the target time point among the N time points on the horizontal axis of the grayscale spectrogram; the acquisition module is specifically used to obtain the current energy grayscale value corresponding to the candidate time point on the horizontal axis of the grayscale spectrogram; the feature fusion module is specifically used to adjust the current energy grayscale value based on the wavelet coefficient corresponding to the target time point, so as to perform image feature fusion of the time domain graph and the grayscale spectrogram.

[0107] Optionally, the determination module 502 is further specifically used to: normalize the wavelet coefficient corresponding to the target time point to obtain the normalized wavelet coefficient corresponding to the target time point; determine the product of the current energy grayscale value and the normalized wavelet coefficient corresponding to the target time point as the adjustment amount of the grayscale value; determine the sum of the current energy grayscale value and the adjustment amount as the target energy grayscale value; the feature fusion module is further specifically used to adjust the current energy grayscale value corresponding to the candidate time point to the target energy grayscale value.

[0108] Optionally, the determination module 502 is further specifically used to input the fused sound feature into a sound classification model to determine the abnormal sound type of the sound signal to be tested. The sound classification model is used to determine the abnormal type of the sound signal. The sound classification model is trained by multiple abnormal sound signals and sample abnormal types corresponding to each abnormal sound signal.

[0109] Figure 6 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application.

[0110] For example, Figure 6 As shown, the vehicle 600 includes: a memory 601 and a processor 602, wherein the memory 601 stores an executable program code 603, and the processor 602 is used to call and execute the executable program code 603 to perform a method for identifying abnormal sound types.

[0111] In addition, an embodiment of the present application also protects a device, which may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to perform a method for identifying abnormal sound types provided in an embodiment of the present application.

[0112] In this embodiment, the functional modules of the device can be divided according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0113] In the case of dividing each functional module according to each function, the device may also include a collection module, a determination module, an acquisition module, a feature fusion module, etc. It should be noted that all relevant contents involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0114] It should be understood that the device provided in this embodiment is used to execute the above-mentioned method for identifying the type of abnormal sound, and thus can achieve the same effect as the above-mentioned implementation method.

[0115] In the case of an integrated unit, the device may include a processing module and a storage module. When the device is applied to a vehicle, the processing module may be used to control and manage the actions of the vehicle. The storage module may be used to support the vehicle in executing related executable program codes, etc.

[0116] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits shown in conjunction with the disclosure of the present application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0117] In addition, the device provided in the embodiments of the present application may specifically be a chip, a component or a module, and the chip may include a connected processor and a memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute a method for identifying abnormal sound types provided in the above embodiments.

[0118] This embodiment also provides a computer-readable storage medium, in which executable program code is stored. When the executable program code is run on a computer, the computer executes the above-mentioned related method steps to implement a method for identifying abnormal sound types provided in the above embodiment.

[0119] This embodiment further provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement a method for identifying abnormal sound types provided in the above embodiment.

[0120] Among them, the device, computer-readable storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0121] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0122] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0123] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for identifying abnormal sound types, characterized in that: The method comprises: Collecting a sound signal to be tested, and determining a spectrogram and a time domain graph of the sound signal to be tested, wherein each time point of N time points of the sound signal to be tested on the horizontal axis of the time domain graph corresponds to an amplitude of the sound signal to be tested on the vertical axis, and N is a positive integer greater than 1; Based on the amplitudes at the N time points, determining a target time point, wherein the target time point is a time point at which the sound signal to be tested is abnormal among the N time points; Based on the target time point and the amplitude at the target time point, fusing the time domain graph and the spectrogram with image features to obtain a fused sound feature; Based on the fused sound feature, the abnormal sound type of the sound signal to be detected is determined.

2. The method according to claim 1, characterized in that The determining the target time point based on the amplitudes of the N time points includes: For each adjacent first time point and second time point among the N time points, determining an absolute value of a difference between an amplitude at the second time point and an amplitude at the first time point; When the absolute value is greater than a preset threshold, the first time point and the second time point are determined as the target time points.

3. The method according to claim 2, characterized in that The method for determining the preset threshold includes: Obtain M abnormal sound signals from the sound library, each of which is marked with an abnormal time point at which the abnormality occurs, and M is a positive integer greater than 1; Based on the abnormal time points, determining a target sound signal from the M abnormal sound signals, wherein the number of abnormal time points corresponding to the target sound signal is greater than a preset number; Performing a wavelet transform on each of the target sound signals to obtain a wavelet coefficient corresponding to each of the P time points of each of the target sound signals, where P is a positive integer greater than 1; The preset threshold is determined based on an average value of differences between wavelet coefficients corresponding to adjacent abnormal time points in the P time points of each of the target sound signals.

4. The method according to claim 1, characterized in that The step of fusing the time domain graph and the spectrogram with image features based on the target time point and the amplitude at the target time point to obtain a fused sound feature includes: grayscale the spectrogram to obtain a grayscale spectrogram; Based on the target time point and the amplitude at the target time point, the time domain image and the grayscale spectrogram are fused with image features to obtain the fused sound feature.

5. The method according to claim 4, characterized in that The amplitude on the vertical axis of the time domain graph is a wavelet coefficient, each time point of the sound signal to be measured on the horizontal axis of the grayscale spectrogram corresponds to an energy grayscale value of the sound signal to be measured, and the image feature fusion of the time domain graph and the grayscale spectrogram based on the target time point and the amplitude at the target time point includes: Determine a candidate time point that matches the target time point among the N time points on the horizontal axis of the grayscale spectrogram; Obtaining the current energy grayscale value corresponding to the candidate time point on the horizontal axis of the grayscale spectrogram; Based on the wavelet coefficient corresponding to the target time point, the current energy grayscale value is adjusted to fuse the image features of the time domain image and the grayscale spectrogram.

6. The method according to claim 5, characterized in that The adjusting the current energy grayscale value based on the wavelet coefficient corresponding to the target time point includes: Normalizing the wavelet coefficients corresponding to the target time point to obtain normalized wavelet coefficients corresponding to the target time point; The product of the current energy grayscale value and the normalized wavelet coefficient corresponding to the target time point is determined as the grayscale value adjustment amount; Determine the sum of the current energy grayscale value and the adjustment amount as the target energy grayscale value; The current energy grayscale value corresponding to the candidate time point is adjusted to the target energy grayscale value.

7. The method according to any one of claims 1 to 6, characterized in that The determining the abnormal sound type of the sound signal to be detected based on the fused sound feature includes: The fused sound feature is input into a sound classification model to determine the abnormal sound type of the sound signal to be tested. The sound classification model is used to determine the abnormal type of the sound signal. The sound classification model is trained by using multiple abnormal sound signals and sample abnormal types corresponding to each of the abnormal sound signals.

8. A device for identifying abnormal sound types, characterized in that: The device comprises: An acquisition module, used for acquiring a sound signal to be tested, and determining a spectrogram and a time domain diagram of the sound signal to be tested, wherein each time point of N time points of the sound signal to be tested on the horizontal axis of the time domain diagram corresponds to an amplitude of the sound signal to be tested on the vertical axis, and N is a positive integer greater than 1; Identify modules for: Based on the amplitudes at the N time points, determining a target time point, wherein the target time point is a time point at which the sound signal to be tested is abnormal among the N time points; Based on the target time point and the amplitude at the target time point, fusing the time domain graph and the spectrogram with image features to obtain a fused sound feature; Based on the fused sound feature, the abnormal sound type of the sound signal to be detected is determined.

9. A vehicle, characterized in that: The vehicle comprises: A memory for storing executable program codes; A processor, configured to call and run the executable program code from the memory, so that the vehicle executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an executable program code, and when the executable program code is executed, the method according to any one of claims 1 to 7 is implemented.

Images