Firecracker detection method, firecracker detection model training method, firecracker detection device, firecracker detection equipment and medium

Abstract

The invention discloses a firecracker detection method, a firecracker detection model training method, a firecracker detection device, firecracker detection equipment and a medium. The method comprises the following steps: performing feature extraction on a collected audio signal to obtain a feature vector; performing aperture classification on the feature vectors to obtain aperture types; performing waveform detection on the feature vector to obtain a propagation shock wave signal and a diffusion shock wave signal; and according to the aperture type, the propagation shock wave signal and the diffusion shock wave signal, judging that the firecracker audio exists in the acquired audio signal and the aperture type of the existing firecracker. According to the embodiment of the invention, the firecracker detection accuracy can be improved.

Description

Technical Field

[0001] This invention relates to the field of Internet technology, and in particular to firecracker detection, firecracker detection model training methods, devices, equipment and media. Background Technology

[0002] To facilitate the safety supervision of fireworks and firecrackers in cities, research related to firecracker testing is increasing year by year.

[0003] Traditional firecracker detection methods only consider the relationship between the characteristics of the acquired audio signal and the firecracker's caliber, focusing on extracting obvious signal features to determine the caliber. However, in urban areas, significant environmental noise interference hinders signal feature extraction. Furthermore, relying solely on the relationship between the acquired audio signal characteristics and the firecracker's caliber for detection results in a single standard and low accuracy. Summary of the Invention

[0004] This invention provides a method, apparatus, equipment, and medium for firecracker detection and firecracker detection model training, which can improve the accuracy of firecracker detection.

[0005] According to one aspect of the present invention, an embodiment of the present invention provides a method for detecting firecrackers, the method comprising:

[0006] Feature vectors are obtained by extracting features from the acquired audio signals;

[0007] The feature vectors are classified by caliber to obtain the caliber type;

[0008] Waveform detection is performed on the feature vector to obtain the propagating shock wave signal and the spreading shock wave signal;

[0009] Based on the caliber type, propagating shock wave signal, and diffused shock wave signal, it is determined whether the collected audio signal contains firecracker audio and the caliber type of the firecracker.

[0010] According to one aspect of the present invention, an embodiment of the present invention provides a method for training a firecracker detection model, the method comprising:

[0011] Acquire training samples, which include: sample signals, true value propagating shock wave signals, true value spreading shock wave signals, and true value aperture; the sample signals are generated by adding random interference signals to the true value firecracker audio signals; the true value firecracker audio signals are synthesized by the true value propagating shock wave signals and the true value spreading shock wave signals;

[0012] The training samples are then input into the firecracker detection model;

[0013] The model extracts features from the sample signals to obtain feature vectors;

[0014] The feature vectors are classified by caliber to obtain the caliber type;

[0015] Waveform detection is performed on the feature vector to obtain the propagating shock wave signal and the spreading shock wave signal;

[0016] Calculate the first difference between the caliber type and the true caliber;

[0017] Calculate the second difference between the propagated shock signal and the true propagated shock signal;

[0018] Calculate the third difference between the diffused shock wave signal and the true diffused shock wave signal;

[0019] Calculate the loss value based on the first difference, the second difference, and the third difference;

[0020] Adjust the parameters of the firecracker detection model based on the loss value.

[0021] According to another aspect of the present invention, embodiments of the present invention also provide a firecracker detection device, the device comprising:

[0022] The feature vector acquisition module is used to extract features from the acquired audio signal to obtain feature vectors;

[0023] A caliber classification module is used to classify the feature vector to obtain the caliber type;

[0024] The waveform detection module is used to perform waveform detection on the feature vector to obtain the propagating shock wave signal and the diffused shock wave signal;

[0025] The firecracker detection module is used to determine, based on the caliber type, propagating shock wave signal, and diffused shock wave signal, whether the collected audio signal contains firecracker audio and the caliber type of the firecracker.

[0026] According to another aspect of the present invention, embodiments of the present invention also provide a firecracker detection model training device, the device comprising:

[0027] The sample acquisition module is used to acquire training samples, which include: sample signals, true value propagating shock wave signals, true value spreading shock wave signals, and true value aperture; the sample signals are generated by adding random interference signals to the true value firecracker audio signals; the true value firecracker audio signals are synthesized by the true value propagating shock wave signals and the true value spreading shock wave signals;

[0028] The sample input module is used to input the training samples into the firecracker detection model;

[0029] The feature extraction module is used by the model to extract features from the sample signal to obtain a feature vector;

[0030] A caliber type acquisition module is used to classify the feature vector to obtain the caliber type;

[0031] The waveform type acquisition module is used to perform waveform detection on the feature vector to obtain the propagating shock wave signal and the diffused shock wave signal;

[0032] The first difference calculation module is used to calculate the first difference between the caliber type and the true caliber.

[0033] The second difference calculation module is used to calculate the second difference between the propagating shock wave signal and the true propagating shock wave signal.

[0034] The third difference calculation module is used to calculate the third difference between the diffused shock wave signal and the true diffused shock wave signal.

[0035] The loss value calculation module is used to calculate the loss value based on the first difference, the second difference, and the third difference;

[0036] The parameter adjustment module is used to adjust the parameters of the firecracker detection model according to the loss value.

[0037] According to another aspect of the present invention, embodiments of the present invention also provide a firecracker detection or firecracker detection model training device, the firecracker detection or firecracker detection model training device comprising:

[0038] At least one processor; and

[0039] A memory that is communicatively connected to at least one processor; wherein,

[0040] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to execute the firecracker detection or firecracker detection model training method according to any embodiment of the present invention.

[0041] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the firecracker detection or firecracker detection model training method of any embodiment of the present invention.

[0042] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the firecracker detection or firecracker detection model training method according to any embodiment of the present invention.

[0043] The technical solution of this invention extracts feature vectors from the acquired audio signals, simultaneously completes aperture classification and waveform detection based on the feature vectors, and then integrates the aperture type with the characteristics of propagating shock wave signals and diffused shock wave signals for comprehensive judgment. Through dual verification, it achieves accurate determination of the existence of firecracker audio and the corresponding aperture type, thereby improving the accuracy of firecracker detection and solving the problem of low accuracy caused by relying on a single signal feature in firecracker detection.

[0044] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart of a firecracker detection method provided by an embodiment of the present invention;

[0047] Figure 2 This is a flowchart of a firecracker detection model training method provided by an embodiment of the present invention;

[0048] Figure 3 This is a structural diagram of a firecracker detection device provided according to an embodiment of the present invention;

[0049] Figure 4 This is a structural diagram of a firecracker detection model training device provided according to an embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of the structure of a firecracker detection or firecracker detection model training device provided in the embodiments of the present invention. Detailed Implementation

[0051] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0052] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0053] The acquisition, storage, and application of audio signals and other related technologies in the technical solutions of this invention comply with relevant laws and regulations and do not violate public order and good morals.

[0054] Figure 1 This is a flowchart illustrating a firecracker detection method provided by an embodiment of the present invention. This embodiment is applicable to firecracker detection, and the method can be executed by a firecracker detection device, which can be implemented in hardware and / or software. The firecracker detection device can be configured in a server.

[0055] See Figure 1 The firecracker detection method shown includes:

[0056] S101. Extract features from the acquired audio signal to obtain a feature vector.

[0057] The audio signal collected can be an audio signal from the urban environment. The system determines whether there is a firecracker sound in the urban environment. Parameters reflecting audio characteristics, such as short-time energy and frequency band range, are extracted from the time and / or frequency domains of the collected audio signal, and these parameters are integrated into a feature vector.

[0058] In an optional embodiment, the step of extracting features from the acquired audio signal to obtain a feature vector includes: performing continuous wavelet transform on the acquired audio signal to obtain a wavelet time-spectrum; and extracting features from the wavelet time-spectrum to obtain a feature vector.

[0059] In this context, the wavelet time-spectrum can be a two-dimensional graph describing the energy strength of a signal. The horizontal axis of the wavelet time-spectrum represents time, the vertical axis represents frequency, and the grayscale values ​​in the graph represent the energy intensity of the signal.

[0060] The acquired audio signal is subjected to continuous wavelet transform, and the formula for continuous wavelet transform is as follows:

[0061]

[0062] Among them, W f (a,b) are wavelet coefficients, and the wavelet coefficient W... f The square of the absolute value of (a,b) reflects the signal energy intensity; a is the scale parameter, used to control the frequency resolution; b is the translation parameter, used to control the time translation; f(t) is the acquired audio signal. These are wavelet basis functions, which are used as a reference to compare with the acquired audio signal and obtain the features in the acquired audio signal.

[0063] Within the range of frequency resolution values, the value of 'a' is changed successively by a preset scale change amount to obtain multiple 'a' values, denoted as ai. i Within the time range of audio signal acquisition, the value of b is changed successively by a preset translation amount to obtain multiple b values, denoted as b. j Calculate each group (a) i ,b j The wavelet coefficients W f (a i b j The process involves obtaining a two-dimensional matrix, mapping this matrix to a two-dimensional time-spectrum graph, and finally obtaining the wavelet time-spectrum graph of the acquired audio signal. Feature extraction is then performed on the wavelet time-spectrum graph to obtain feature vectors.

[0064] As can be seen, by performing continuous wavelet transform on the acquired audio signal to obtain the wavelet time spectrum, the time and frequency characteristics of the acquired audio signal are presented synchronously in a two-dimensional form, which can accurately capture the frequency component changes of the signal at different times and improve the recognition accuracy of feature extraction.

[0065] In an optional embodiment, the step of performing continuous wavelet transform on the acquired audio signal to obtain a wavelet time spectrum includes: performing continuous wavelet transform on the acquired audio signal using a first time window to obtain a low-frequency time spectrum; performing continuous wavelet transform on the acquired audio signal using a second time window to obtain a mid-frequency time spectrum; the duration of the first time window is greater than the duration of the second time window; performing continuous wavelet transform on the acquired audio signal using a third time window to obtain a high-frequency time spectrum; the duration of the second time window is greater than the duration of the third time window; and generating a wavelet time spectrum based on the low-frequency time spectrum, the mid-frequency time spectrum, and the high-frequency time spectrum.

[0066] The time window can be the time interval used in continuous wavelet transform to extract the acquired audio signal for local analysis.

[0067] The duration of the time window reflects the magnitude of the scale parameter 'a'. The specific relationship between the duration of the time window and the scale parameter 'a' is as follows: the longer the time window, the wider the time window, and the larger the scale parameter 'a'; the shorter the time window, the narrower the time window, and the smaller the scale parameter 'a'.

[0068] A large scale parameter 'a' stretches the wavelet basis function in the time dimension, resulting in a wider and smoother waveform, which is beneficial for analyzing slow, low-frequency trends. A small scale parameter 'a' compresses the wavelet basis function in the time dimension, resulting in a narrower and steeper waveform, which is beneficial for analyzing details of abrupt changes in high frequencies.

[0069] The first time window can be a relatively long time window. The second time window can be a time window of intermediate length. The third time window can be a relatively short time window. In some embodiments, the first time window is 6ms, the second time window is 3ms, and the third time window is 1ms.

[0070] Continuous wavelet transforms are performed using time windows of varying lengths to obtain time spectrograms of different frequencies. These time spectrograms are then aligned along the time and frequency dimensions, and the energy intensities at the same time and frequency in each time spectrogram are summed to obtain the wavelet time spectrogram.

[0071] As can be seen, by generating wavelet time spectrum diagrams from multiple time windows of different durations, the advantages of analyzing different high and low frequency band signals by different time windows of different durations can be combined to obtain the full-band characteristics of the acquired audio signal.

[0072] S102. Classify the feature vectors by caliber to obtain the caliber type.

[0073] The caliber can refer to the thickness of the firecracker's opening. The caliber reflects the firecracker's explosive capacity. Smaller caliber firecrackers contain less explosive material and have a lower explosive force; larger caliber firecrackers contain more explosive material and have a higher explosive force. Different regions have different restrictions or prohibitions on the time, location, and types of firecrackers. Identifying and classifying firecrackers by caliber helps determine whether the types of firecrackers used in a region comply with regulations, which is beneficial for supervising the compliant use of firecrackers.

[0074] Firecrackers of different calibers exhibit significant differences in the duration and distribution of energy released upon detonation. Larger caliber firecrackers release energy for a longer duration, with the energy concentrated in the low-frequency region; while smaller caliber firecrackers release energy for a shorter duration, with the energy concentrated in the mid-to-high frequency region. Based on these energy differences, the feature vectors are classified to determine the caliber type of the firecrackers.

[0075] The caliber type may include: large caliber or small caliber. In some embodiments, the caliber type includes: 7.62 mm caliber or 12.7 mm caliber.

[0076] S103. Perform waveform detection on the feature vector to obtain the propagating shock wave signal and the diffused shock wave signal.

[0077] Among them, the explosion of firecrackers generates propagating shock waves and spreading shock waves. Propagating shock wave signals are characterized by extremely short rise times and rapid energy decay, while spreading shock wave signals are characterized by longer rise times and slow energy decay. Spreading shock wave signals are often accompanied by obvious oscillating wakes.

[0078] Based on the signal characteristics of propagating shock waves and spreading shock waves, waveform detection is performed on the feature vectors to obtain the propagating shock wave signal and the spreading shock wave signal. In some embodiments, determination features for the propagating shock wave signal and the spreading shock wave signal are set respectively. Optionally, the determination features can be energy intensity ranges or attenuation rate ranges, etc. The feature vectors are matched with the determination features of the propagating shock wave signal. If the match is successful, the propagating shock wave signal in the acquired audio signal is determined based on the matched portion of the feature vector. If the match is unsuccessful, it indicates that there is no propagating shock wave signal in the acquired audio signal. Similarly, the feature vectors are matched with the determination features of the spreading shock wave signal. If the match is successful, the spreading shock wave signal in the acquired audio signal is determined based on the matched portion of the feature vector. If the match is unsuccessful, it indicates that there is no spreading shock wave signal in the acquired audio signal.

[0079] S104. Based on the caliber type, propagating shock wave signal, and diffused shock wave signal, determine whether the collected audio signal contains firecracker audio and the caliber type of the firecracker.

[0080] Specifically, if the waveform detection does not detect a propagating shock wave signal or a spreading shock wave signal, then the acquired audio signal does not contain the sound of firecrackers. If the waveform detection detects both a propagating shock wave signal and a spreading shock wave signal, then the acquired audio signal contains the sound of firecrackers.

[0081] The caliber type obtained from the caliber classification is verified based on the propagation shock wave signal and the diffusion shock wave signal. Large-caliber firecrackers exhibit high intensity of propagation shock wave signal and long propagation distance of diffusion shock wave signal. Small-caliber firecrackers exhibit low intensity of propagation shock wave signal and short propagation distance of diffusion shock wave signal.

[0082] If the intensity of the propagating shock wave signal and the propagation distance of the diffused shock wave signal match the aperture type—that is, when the aperture type is large, the intensity of the propagating shock wave signal is high, and the propagation distance of the diffused shock wave signal is long; when the aperture type is small, the intensity of the propagating shock wave signal is low, and the propagation distance of the diffused shock wave signal is short—then the aperture type is correct. The aperture type obtained from the aperture classification is determined as the aperture type of the firecrackers present in the acquired audio signal. The audio signal containing the firecracker audio and the aperture type of the firecracker are output. If the intensity of the propagating shock wave signal or the propagation distance of the diffused shock wave signal does not match the aperture type, only the audio signal containing the firecracker audio is output, without the firecracker aperture type.

[0083] The technical solution of this invention extracts feature vectors from the acquired audio signals, simultaneously completes aperture classification and waveform detection based on the feature vectors, and then integrates the aperture type with the characteristics of propagating shock wave signals and diffused shock wave signals for comprehensive judgment. Through dual verification, it achieves accurate determination of the existence of firecracker audio and the corresponding aperture type, improves the accuracy of firecracker detection, and solves the problem of low accuracy caused by firecracker detection relying on a single signal feature.

[0084] Figure 2 This is a flowchart illustrating a method for training a firecracker detection model according to an embodiment of the present invention. This embodiment is applicable to firecracker detection model training, and the method can be executed by a firecracker detection model training device, which can be implemented in hardware and / or software. This firecracker detection model training device can be configured in a server. It should be noted that parts not described in detail in this embodiment can be found in the descriptions of other embodiments.

[0085] See Figure 2 The firecracker detection model training method shown includes:

[0086] S201. Obtain training samples, the training samples including: sample signals, true value propagating shock wave signals, true value spreading shock wave signals and true value aperture; the sample signals are generated by adding random interference signals to the true value firecracker audio signals; the true value firecracker audio signals are synthesized by the true value propagating shock wave signals and the true value spreading shock wave signals.

[0087] Here, the true value propagating shock wave signal can be a propagating shock wave signal labeled with the true value. The true value spreading shock wave signal can be a spreading shock wave signal labeled with the true value. The true value refers to the real and correct output result corresponding to the input data in model training.

[0088] The true aperture can be the actual, correct aperture of a firecracker corresponding to the true firecracker audio signal. True propagating shock wave signals and true spreading shock wave signals contain multiple aperture types. True propagating shock wave signals and true spreading shock wave signals of the same aperture type are combined to form the true firecracker signal. The firecracker aperture corresponding to the true firecracker signal is the aperture type of the true propagating shock wave signal and the true spreading shock wave signal.

[0089] In some implementation examples, the caliber types include large-caliber and small-caliber. The true value propagating shock wave signal includes the true value propagating shock wave signal of both large-caliber and small-caliber firecrackers. The true value diffused shock wave signal includes the true value diffused shock wave signal of both large-caliber and small-caliber firecrackers. The true value propagating shock wave signal and the true value diffused shock wave signal of the large-caliber firecracker are combined to form the true value audio signal of the large-caliber firecracker, whose true value caliber is large-caliber. The true value propagating shock wave signal and the true value diffused shock wave signal of the small-caliber firecracker are combined to form the true value audio signal of the small-caliber firecracker, whose true value caliber is small-caliber.

[0090] Random interference signals are added to the true audio signal of the firecracker to generate sample signals. The true aperture of the true audio signal of the firecracker is used as the true aperture of the sample signal.

[0091] In an optional embodiment, obtaining training samples includes: obtaining a true value propagating shock wave signal, a true value spreading shock wave signal, and a true value aperture; superimposing the true value propagating shock wave signal and the true value spreading shock wave signal to obtain a true value firecracker audio signal; randomly selecting consecutive time periods multiple times from the true value firecracker audio signal, setting the amplitude of the signal in that time period to zero to obtain multiple first signals; and generating training samples for each first signal by combining the first signal, the true value propagating shock wave signal, the true value spreading shock wave signal, and the true value aperture.

[0092] The amplitudes of the propagating shock wave signal and the diffused shock wave signal at the same time point are summed to obtain the superimposed signal amplitude at the corresponding time. The superimposed signal amplitudes at all times are integrated in chronological order to obtain the true firecracker audio signal.

[0093] To simulate signal loss due to transmission interruption, equipment failure, or obstruction in a real-world environment, random interference signals are added to the true firecracker audio signal.

[0094] Randomly select a continuous time period from the true value firecracker audio signal. Optionally, the duration of the time period is set to 10% of the total duration of the true value firecracker audio signal. Set the signal amplitude within this time period to zero to obtain the first signal.

[0095] By randomly selecting consecutive time periods from the true firecracker audio signal and setting the signal amplitude to zero within each time period, multiple first signals are obtained.

[0096] For each first signal, a first signal training sample is generated, containing the first signal, a true value propagating shock wave signal, a true value spreading shock wave signal, and a true value aperture. The true value propagating shock wave signal and the true value spreading shock wave signal from the first signal training sample are superimposed to obtain the true value firecracker audio signal used to generate the first signal, and the true value aperture is the true value aperture of the first signal.

[0097] It can be seen that by superimposing the true value propagating shock wave signal and the true value diffused shock wave signal, and by randomly selecting consecutive time periods from the true value firecracker audio signal and setting the amplitude of the signal in those time periods to zero, it is possible to simulate the situation where the signal is partially lost in the actual environment. This can train the model to recognize incomplete signals and improve the model's recognition ability.

[0098] In an optional embodiment, acquiring training samples includes: acquiring a true value propagating shock wave signal, a true value spreading shock wave signal, and a true value aperture; superimposing the true value propagating shock wave signal and the true value spreading shock wave signal to obtain a true value firecracker audio signal; determining at least one start signal end endpoint and at least one end signal start endpoint in the true value firecracker audio signal; arranging and combining each of the start signal end endpoints and the end signal start endpoints to obtain multiple combinations; for each combination, adjusting the amplitude of the start signal unit formed by the start signal end endpoint and the start endpoint of the true value firecracker audio signal in the combination so that the amplitude of the start endpoint of the adjusted start signal unit is zero; adjusting the amplitude of the end signal unit formed by the end signal start endpoint and the end endpoint of the true value firecracker audio signal in the combination so that the amplitude of the end endpoint of the adjusted end signal unit is zero; determining the adjusted true value firecracker audio signal of the same combination as a second signal; and generating training samples for the second signal by combining the second signal, the true value propagating shock wave signal, the true value spreading shock wave signal, and the true value aperture.

[0099] In this process, random interference signals are added to the true firecracker audio signal to simulate the gradual changes in signal strength caused by changes in distance, equipment movement, or environmental obstruction during actual monitoring.

[0100] The process of simulating the gradual change of the signal causes the amplitude of the true firecracker audio signal to increase linearly from zero to the amplitude of the initial signal; and causes the amplitude of the true firecracker audio signal to decrease linearly from the amplitude of the final signal to zero.

[0101] Specifically, on the true value firecracker audio signal, at least one start signal end point is selected near the start point of the true value firecracker audio signal, and at least one end signal start point is selected near the end point of the true value firecracker audio signal. The start signal end points and end signal start points are randomly combined to generate multiple sets of true value firecracker audio signals to be processed.

[0102] The signal segment starting from the beginning of the true-value firecracker audio signal and ending at the end of the starting signal is designated as the starting signal unit. The amplitude of the starting signal unit is adjusted to be an increasing signal segment with a starting amplitude of zero and an ending amplitude equal to the original signal amplitude. Optionally, the starting signal unit is divided into several smaller signal units along the time dimension. The amplitudes of these smaller signal units are then multiplied sequentially by the corresponding term number in an arithmetic sequence, in ascending order of time. The results are used to update the amplitude of each smaller signal unit. For example, the amplitude of the first smaller signal unit is multiplied by the first term of the sequence. The first term of the arithmetic sequence is 0, the last term is 1, and the number of terms in the arithmetic sequence matches the number of smaller signal units. Finally, the updated smaller signal units are integrated and smoothly transitioned.

[0103] The signal segment starting from the beginning of the ending signal and ending at the end of the true firecracker audio signal is designated as the ending signal unit. The amplitude of the ending signal unit is adjusted to be a decreasing signal segment with the starting amplitude being the original signal amplitude and the ending amplitude being zero. Optionally, the ending signal unit is divided into several smaller signal units along the time dimension. The amplitudes of these smaller signal units are multiplied sequentially by the corresponding term number in an arithmetic progression, in ascending order of time. The results are used to update the amplitude of each smaller signal unit. For example, the amplitude of the first smaller signal unit is multiplied by the first term of the progression. The first term of the arithmetic progression is 1, the last term is 0, and the number of terms in the arithmetic progression matches the number of smaller signal units. Finally, the updated smaller signal units are integrated and smoothly transitioned.

[0104] After processing by both the start signal unit and the end signal unit, the second signal is obtained.

[0105] For each second signal, a first signal training sample is generated, containing the second signal, the true value propagating shock wave signal, the true value spreading shock wave signal, and the true value aperture. The true value propagating shock wave signal and the true value spreading shock wave signal from the second signal training sample are superimposed to obtain the true value firecracker audio signal used to generate the second signal, with the true value aperture being the true value aperture of the second signal.

[0106] It is evident that by determining the end endpoint of each starting signal and the start endpoint of each ending signal, and adjusting the amplitude of the starting signal unit and the ending signal unit, the gradual change in signal strength caused by distance changes, equipment movement, or environmental obstruction in actual monitoring can be simulated. This helps to specifically improve the model's adaptability in complex environments and provides the model with the ability to identify real acquired signals.

[0107] S202. Input the training samples into the firecracker detection model.

[0108] The model can be a convolutional neural network model. Optionally, the model is a three-layer two-dimensional convolutional neural network model. Optionally, the sample signal, the true value propagating shock wave signal, and the true value spreading shock wave signal in the training samples are subjected to continuous wavelet transform to obtain wavelet time-frequency spectra. The wavelet time-frequency spectra of the sample signal, the true value propagating shock wave signal, the true value spreading shock wave signal, and the true value aperture are input into the model.

[0109] S203. The model extracts features from the sample signal to obtain a feature vector.

[0110] In this process, feature extraction is performed on the sample signal to obtain the feature vector of the sample signal.

[0111] S204. Classify the feature vectors by caliber to obtain the caliber type.

[0112] In this process, the feature vectors of the sample signals are classified by aperture to obtain the aperture classification model classification result, i.e., aperture type.

[0113] S205. Perform waveform detection on the feature vector to obtain the propagating shock wave signal and the diffused shock wave signal.

[0114] In this process, waveform detection is performed on the feature vector of the sample signal to obtain the waveform detection model detection result, i.e., the waveform is a propagating shock wave signal or a diffused shock wave signal.

[0115] S206. Calculate the first difference between the caliber type and the true caliber.

[0116] The classification results of the caliber classification model are compared with the actual caliber, and the classification error of the model is calculated. The classification error of the model is used as the first difference.

[0117] S207. Calculate the second difference between the propagating shock signal and the true propagating shock signal.

[0118] Specifically, the propagation shock wave signal in the model detection results is compared with the true propagation shock wave signal to calculate the model's propagation shock wave signal detection error. This propagation shock wave signal detection error is used as the second difference.

[0119] S208. Calculate the third difference between the diffused shock wave signal and the true diffused shock wave signal.

[0120] Specifically, the diffusion shock wave signal in the model's detection results is compared with the true propagation shock wave signal to calculate the model's diffusion shock wave signal detection error. This diffusion shock wave signal detection error is considered as the third difference.

[0121] S209. Calculate the loss value based on the first difference, the second difference, and the third difference.

[0122] The loss value is obtained by calculating the first, second, and third differences. Optionally, the loss value is obtained by weighted summing of the first, second, and third differences.

[0123] S210. Adjust the parameters of the firecracker detection model according to the loss value.

[0124] The training process is complete when the loss value converges or reaches its minimum. Alternatively, training is complete when the model's accuracy on the validation set is greater than or equal to a preset accuracy threshold. Model parameters are adjusted based on the loss value to improve the model's recognition accuracy. The trained firecracker detection model can then be used to implement firecracker detection methods.

[0125] The technical solution of this invention will obtain training samples and input them into the model to perform caliber classification and waveform detection. The waveform reflects the physical phenomenon of firecracker explosion, and the caliber reflects the energy scale under this physical phenomenon. By using the same model for caliber classification and waveform detection, the model can learn the inherent correlation between waveform features and caliber. The mutual guidance of the two types of tasks improves the recognition accuracy of the model.

[0126] Figure 3 This is a schematic diagram of a firecracker detection device provided in an embodiment of the present invention. This embodiment of the present invention is applicable to firecracker detection, and the device can execute a firecracker detection method. The device can be implemented in hardware and / or software.

[0127] See Figure 3 The firecracker detection device shown includes:

[0128] The feature vector acquisition module 301 is used to extract features from the acquired audio signal to obtain a feature vector;

[0129] Aperture classification module 302 is used to classify the feature vector to obtain aperture type;

[0130] Waveform detection module 303 is used to perform waveform detection on the feature vector to obtain the propagating shock wave signal and the diffused shock wave signal;

[0131] The firecracker detection module 304 is used to determine, based on the caliber type, propagating shock wave signal, and diffused shock wave signal, whether the collected audio signal contains firecracker audio and the caliber type of the firecracker.

[0132] The technical solution of this invention extracts feature vectors from the acquired audio signals, simultaneously completes aperture classification and waveform detection based on the feature vectors, and then integrates the aperture type with the characteristics of propagating shock wave signals and diffused shock wave signals for comprehensive judgment. Through dual verification, it achieves accurate determination of the existence of firecracker audio and the corresponding aperture type, improves the accuracy of firecracker detection, and solves the problem of low accuracy caused by firecracker detection relying on a single signal feature.

[0133] In an optional embodiment, the feature vector acquisition module 301 includes:

[0134] The wavelet time-spectrum acquisition unit is used to perform continuous wavelet transform on the acquired audio signal to obtain the wavelet time-spectrum.

[0135] The feature vector acquisition unit is used to extract features from the wavelet time-spectrum map to obtain feature vectors.

[0136] In an optional embodiment, the wavelet time-spectrum acquisition unit includes:

[0137] The low-frequency time spectrum acquisition subunit is used to perform continuous wavelet transform on the acquired audio signal using the first time window to obtain the low-frequency time spectrum.

[0138] The intermediate frequency time-spectrum acquisition subunit is used to perform continuous wavelet transform on the acquired audio signal using a second time window to obtain the intermediate frequency time-spectrum; the duration of the first time window is greater than the duration of the second time window;

[0139] The high-frequency time-spectrum acquisition subunit is used to perform continuous wavelet transform on the acquired audio signal using a third time window to obtain a high-frequency time-spectrum; the duration of the second time window is greater than the duration of the third time window;

[0140] The wavelet time-spectrum acquisition sub-unit is used to generate a wavelet time-spectrum based on the low-frequency time-spectrum, mid-frequency time-spectrum, and high-frequency time-spectrum.

[0141] The firecracker detection device provided in this embodiment of the invention can execute the firecracker detection method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the firecracker detection method.

[0142] Figure 4This is a schematic diagram of a firecracker detection model training device provided in an embodiment of the present invention. This embodiment of the present invention is applicable to the training of firecracker detection models. The device can execute firecracker detection model training methods and can be implemented in hardware and / or software.

[0143] See Figure 4 The firecracker detection model training device shown includes:

[0144] The sample acquisition module 401 is used to acquire training samples, which include: sample signals, true value propagating shock wave signals, true value spreading shock wave signals, and true value aperture; the sample signals are generated by adding random interference signals to the true value firecracker audio signals; the true value firecracker audio signals are synthesized by the true value propagating shock wave signals and the true value spreading shock wave signals.

[0145] The sample input module 402 is used to input the training samples into the firecracker detection model;

[0146] Feature extraction module 403 is used by the model to extract features from the sample signal to obtain a feature vector;

[0147] Aperture type acquisition module 404 is used to classify the feature vector to obtain the aperture type;

[0148] The waveform type acquisition module 405 is used to perform waveform detection on the feature vector to obtain the propagating shock wave signal and the diffused shock wave signal;

[0149] The first difference calculation module 406 is used to calculate the first difference between the caliber type and the true caliber.

[0150] The second difference calculation module 407 is used to calculate the second difference between the propagating shock wave signal and the true propagating shock wave signal.

[0151] The third difference calculation module 408 is used to calculate the third difference between the diffused shock wave signal and the true value diffused shock wave signal.

[0152] The loss value calculation module 409 is used to calculate the loss value based on the first difference, the second difference, and the third difference;

[0153] The parameter adjustment module 410 is used to adjust the parameters of the firecracker detection model according to the loss value.

[0154] The technical solution of this invention involves acquiring training samples and inputting them into a model for caliber classification and waveform detection. The waveform reflects the physical phenomenon of firecracker explosion, while the caliber reflects the energy scale under this physical phenomenon. By using the same model for caliber classification and waveform detection, the model can learn the inherent correlation between waveform features and caliber. The mutual guidance between the two tasks improves the model's recognition accuracy.

[0155] In an optional embodiment, the sample acquisition module 401 includes:

[0156] The first signal acquisition unit is used to acquire the true value propagating shock wave signal, the true value spreading shock wave signal, and the true value aperture.

[0157] The first signal superposition unit is used to superimpose the true value propagating shock wave signal and the true value spreading shock wave signal to obtain the true value firecracker audio signal.

[0158] The signal masking unit is used to randomly select consecutive time periods multiple times from the true firecracker audio signal, set the amplitude of the signal during the time period to zero, and obtain multiple first signals.

[0159] The first sample generation unit is used to generate training samples of the first signal for each first signal by taking the first signal, the true value propagating shock wave signal, the true value spreading shock wave signal, and the true value aperture.

[0160] In an optional embodiment, the sample acquisition module 401 includes:

[0161] The second signal acquisition unit is used to acquire the true value propagating shock wave signal, the true value spreading shock wave signal, and the true value aperture.

[0162] The second signal superposition unit is used to superimpose the true value propagating shock wave signal and the true value spreading shock wave signal to obtain the true value firecracker audio signal.

[0163] An endpoint determination unit is used to determine at least one start signal end endpoint and at least one end signal start endpoint in the true firecracker audio signal;

[0164] The signal combination unit is used to arrange and combine the end endpoints of each of the start signals and the start endpoints of each of the end signals to obtain multiple combinations;

[0165] The first gradient addition unit is used to adjust the amplitude of the starting signal unit formed by the end point of the starting signal and the starting point of the true firecracker audio signal in each of the combinations, so that the amplitude of the starting point of the adjusted starting signal unit is zero.

[0166] The second gradient addition unit is used to adjust the amplitude of the end signal unit formed by the start endpoint of the end signal in the combination and the end endpoint of the true firecracker audio signal, so that the amplitude of the end endpoint of the adjusted end signal unit is zero.

[0167] The signal determination unit is used to determine the true value firecracker audio signal after the same combination adjustment as a second signal;

[0168] The second sample generation unit is used to generate training samples of the second signal for each second signal by taking the second signal, the true value propagating shock wave signal, the true value diffused shock wave signal and the true value aperture.

[0169] The firecracker detection device provided in this embodiment of the invention can execute the firecracker detection method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the firecracker detection method.

[0170] Figure 5 A schematic diagram of the structure of a firecracker detection or firecracker detection model training device 500, which can be used to implement embodiments of the present invention, is shown.

[0171] like Figure 5 As shown, the firecracker detection or firecracker detection model training device 500 includes at least one processor 501 and a memory, such as a read-only memory 502 or a random access memory 503, communicatively connected to the at least one processor 501. The memory stores computer programs executable by the at least one processor. The processor 501 can perform various appropriate actions and processes based on the computer program stored in the read-only memory 502 or loaded from the storage unit 508 into the random access memory 503. The random access memory 503 can also store various programs and data required for the operation of the firecracker detection device 500. The processor 501, read-only memory 502, and random access memory 503 are interconnected via a bus 504. An input / output interface 505 is also connected to the bus 504.

[0172] Multiple components in the firecracker detection or firecracker detection model training device 500 are connected to the input / output interface 505, including: an input unit 506, such as a keyboard, mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, optical disk, etc.; and a communication unit 509, such as a network card, modem, wireless transceiver, etc. The communication unit 509 allows the firecracker detection device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0173] Processor 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 501 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 501 performs the various methods and processes described above, such as firecracker detection or firecracker detection model training methods.

[0174] In some embodiments, the firecracker detection or firecracker detection model training method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the firecracker detection or firecracker detection model training device 500 via read-only memory 502 and / or communication unit 509. When the computer program is loaded into random access memory 503 and executed by processor 501, one or more steps of the firecracker detection or firecracker detection model training method described above may be performed. Alternatively, in other embodiments, processor 501 may be configured to perform the firecracker detection or firecracker detection model training method by any other suitable means (e.g., by means of firmware).

[0175] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), systems-on-a-chip (SoCs), complex programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0176] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0177] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, flash memory, optical fiber, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0178] To provide user interaction, the systems and techniques described herein can be implemented on the operational detection device, which includes: a display device (e.g., a cathode ray tube or liquid crystal display monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the firecracker detection or firecracker detection model training device. Other types of devices can also be used to provide user interaction; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0179] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0180] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product within the cloud computing service system. This addresses the shortcomings of traditional physical hosts and virtual private servers, such as high management difficulty and weak business scalability.

[0181] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0182] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting firecrackers, characterized in that, The method includes: Feature vectors are obtained by extracting features from the acquired audio signals; The feature vectors are classified by caliber to obtain the caliber type; Waveform detection is performed on the feature vector to obtain the propagating shock wave signal and the spreading shock wave signal; Based on the caliber type, propagating shock wave signal, and diffused shock wave signal, it is determined whether the collected audio signal contains firecracker audio and the caliber type of the firecracker.

2. The method according to claim 1, characterized in that, The step of extracting features from the acquired audio signal to obtain a feature vector includes: Perform continuous wavelet transform on the acquired audio signal to obtain the wavelet time-spectrum. Feature extraction is performed on the wavelet time-spectrum to obtain the feature vector.

3. The method according to claim 2, characterized in that, The step of performing continuous wavelet transform on the acquired audio signal to obtain the wavelet time-spectrum includes: Using the first time window, continuous wavelet transform is performed on the acquired audio signal to obtain the low-frequency time spectrum. A second time window is used to perform continuous wavelet transform on the acquired audio signal to obtain the intermediate frequency time spectrum; the duration of the first time window is longer than the duration of the second time window. A third time window is used to perform continuous wavelet transform on the acquired audio signal to obtain a high-frequency time spectrum; the duration of the second time window is longer than the duration of the third time window. A wavelet time spectrum is generated based on the low-frequency time spectrum, mid-frequency time spectrum, and high-frequency time spectrum.

4. A method for training a firecracker detection model, characterized in that, The method includes: Acquire training samples, which include: sample signals, true value propagating shock wave signals, true value spreading shock wave signals, and true value aperture; the sample signals are generated by adding random interference signals to the true value firecracker audio signals; the true value firecracker audio signals are synthesized by the true value propagating shock wave signals and the true value spreading shock wave signals; The training samples are then input into the firecracker detection model; The model extracts features from the sample signals to obtain feature vectors; The feature vectors are classified by caliber to obtain the caliber type; Waveform detection is performed on the feature vector to obtain the propagating shock wave signal and the spreading shock wave signal; Calculate the first difference between the caliber type and the true caliber; Calculate the second difference between the propagated shock signal and the true propagated shock signal; Calculate the third difference between the diffused shock wave signal and the true diffused shock wave signal; Calculate the loss value based on the first difference, the second difference, and the third difference; Adjust the parameters of the firecracker detection model based on the loss value.

5. The method according to claim 4, characterized in that, The acquisition of training samples includes: Obtain the true value propagating shock wave signal, the true value diffused shock wave signal, and the true value aperture; The true value propagating shock wave signal and the true value spreading shock wave signal are superimposed to obtain the true value firecracker audio signal; For the true firecracker audio signal, multiple consecutive time periods are randomly selected, and the amplitude of the signal during that time period is set to zero to obtain multiple first signals; For each first signal, the first signal, the true value propagating shock wave signal, the true value spreading shock wave signal, and the true value aperture are used to generate training samples for the first signal.

6. The method according to claim 4, characterized in that, The acquisition of training samples includes: Obtain the true value propagating shock wave signal, the true value diffused shock wave signal, and the true value aperture; The true value propagating shock wave signal and the true value spreading shock wave signal are superimposed to obtain the true value firecracker audio signal; In the true value firecracker audio signal, at least one start signal end point and at least one end signal start point are determined; The end points of each of the starting signals and the start points of each of the ending signals are arranged and combined to obtain multiple combinations; For each of the aforementioned combinations, the amplitude of the starting signal unit formed by the end point of the starting signal and the starting point of the true firecracker audio signal in the combination is adjusted so that the amplitude of the starting point of the adjusted starting signal unit is zero. The amplitude of the end signal unit formed by the start endpoint of the end signal in the combination and the end endpoint of the true firecracker audio signal is adjusted so that the amplitude of the end endpoint of the adjusted end signal unit is zero. The true value firecracker audio signal adjusted by the same combination is determined as a second signal; For each second signal, the second signal, the true value propagating shock wave signal, the true value spreading shock wave signal, and the true value aperture are used to generate training samples for the second signal.

7. A firecracker detection device, characterized in that, The device includes: The feature vector acquisition module is used to extract features from the acquired audio signal to obtain feature vectors; A caliber classification module is used to classify the feature vector to obtain the caliber type; The waveform detection module is used to perform waveform detection on the feature vector to obtain the propagating shock wave signal and the diffused shock wave signal; The firecracker detection module is used to determine, based on the caliber type, propagating shock wave signal, and diffused shock wave signal, whether the collected audio signal contains firecracker audio and the caliber type of the firecracker.

8. A firecracker detection model training device, characterized in that, The device includes: The sample acquisition module is used to acquire training samples, which include: sample signals, true value propagating shock wave signals, true value spreading shock wave signals, and true value aperture; the sample signals are generated by adding random interference signals to the true value firecracker audio signals; the true value firecracker audio signals are synthesized by the true value propagating shock wave signals and the true value spreading shock wave signals; The sample input module is used to input the training samples into the firecracker detection model; The feature extraction module is used by the model to extract features from the sample signal to obtain a feature vector; A caliber type acquisition module is used to classify the feature vector to obtain the caliber type; The waveform type acquisition module is used to perform waveform detection on the feature vector to obtain the propagating shock wave signal and the diffused shock wave signal; The first difference calculation module is used to calculate the first difference between the caliber type and the true caliber. The second difference calculation module is used to calculate the second difference between the propagating shock wave signal and the true propagating shock wave signal. The third difference calculation module is used to calculate the third difference between the diffused shock wave signal and the true diffused shock wave signal. The loss value calculation module is used to calculate the loss value based on the first difference, the second difference, and the third difference; The parameter adjustment module is used to adjust the parameters of the firecracker detection model according to the loss value.

9. A firecracker detection or firecracker detection model training device, characterized in that, The firecracker detection or firecracker detection model training device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the firecracker detection or firecracker detection model training method according to any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the firecracker detection or firecracker detection model training method according to any one of claims 1-6.

Images