Panic pedestrian squeal behavior audio feature recognition method and device

Abstract

The invention relates to a panic pedestrian screaming behavior audio feature recognition method and device. The method comprises the following steps: extracting an audio part from a video; intercepting audio features of a first frequency band in the audio, and calculating to obtain audio energy distribution; dividing audio energy according to intervals, and establishing a panic behavior audio feature recognition matrix model; and determining the behavior type and gender in the audio according to the audio feature recognition matrix model. Compared with the prior art, the squeal behavior of the panic pedestrian is recognized by analyzing the audio features and establishing the panic behavior audio feature recognition matrix model, and the method has robustness and environmental adaptability.

Description

Technical Field

[0001] The present invention relates to the field of crowd flow stability analysis, and in particular to a method and device for recognizing audio features of screaming behavior of panicked pedestrians. Background Art

[0002] With the increase in the coverage of network surveillance cameras in public places and the richness of audio data, human pedestrian behavior recognition technology based on audio features has been developed and applied in the field of crowd stability research in public places. Domestic and foreign scholars have begun to study crowd evacuation and emergencies in the early 20th century. In various public place emergencies, pedestrians are easily affected by the environment and fall into panic, which in turn triggers various panic behaviors. In view of this, this paper is based on the existing audio feature recognition technology, considers the audio content that may appear in different emergencies, analyzes the audio features of pedestrians' screams in accidents, applies scream audio feature recognition to panic crowd behavior analysis and disturbance propagation research, and establishes a crowd panic behavior audio feature recognition model. It lays the foundation for subsequent exploration of the dynamic impact of panic behavior disturbances on crowd stability and provides new ideas for the study of crowd flow safety in public places.

[0003] There are still some deficiencies in the current research:

[0004] 1. At present, research on pedestrian audio feature recognition rarely considers the situation where the audio content is distorted or incomplete under the influence of panic emotions, and lacks more robust and environmentally adaptable audio feature judgment criteria for panic behavior.

[0005] 2. Most of the existing research on panic crowd behavior analysis based on deep learning methods is based on the movement characteristics of the crowd's limbs. Few studies have been conducted to analyze panic behavior by extracting audio features such as sound frequency, sound pitch or semantic content from audio data. Summary of the invention

[0006] The purpose of the present invention is to provide a method and device for identifying the audio features of panicked pedestrians' screaming behavior. By analyzing the audio features and establishing a panicked behavior audio feature recognition matrix model, the screaming behavior of panicked pedestrians is identified, which is robust and environmentally adaptable.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] A method for identifying audio features of screaming behavior of panicked pedestrians, the method comprising:

[0009] Extract the audio portion from the video;

[0010] Intercepting the audio features of the first frequency band in the audio, and calculating and obtaining the audio energy distribution;

[0011] Divide the audio energy into intervals and establish a matrix model for panic behavior audio feature recognition;

[0012] Determine the behavior type and gender in the audio based on the audio feature recognition matrix model.

[0013] Furthermore, the first frequency band is a 0-750 Hz frequency band.

[0014] Furthermore, the process of intercepting the audio features of the first frequency band in the audio and calculating the audio energy distribution includes:

[0015] Perform Fourier transform on the audio data to obtain the waveform and spectrum of the audio signal;

[0016] Extracting audio features of a first frequency band from the frequency spectrum of the audio signal;

[0017] The audio energy of the first frequency band is normalized to calculate the audio energy distribution.

[0018] Furthermore, the process of dividing the audio energy into intervals includes:

[0019] Get the panic pedestrian screaming behavior audio dataset;

[0020] Randomly select male panic behavior audio and female panic behavior audio from the dataset and analyze the energy distribution of the first frequency band audio;

[0021] Draw the energy normal distribution fitting diagram of male panic behavior audio and female panic behavior audio, and calculate the energy mean and energy standard deviation;

[0022] The audio energy is divided into intervals based on the normal distribution fitting diagram of audio energy of different genders, energy mean and energy standard deviation.

[0023] Furthermore, the audio energy division intervals include: [0, 0.009], [0.009, 0.053], [0.053, 0.076], [0.076, 0.172], [0.172, 0.178], [0.178, 0.206], [0.206, 0.634] and [0.634, 1].

[0024] Furthermore, the process of establishing the panic behavior audio feature recognition matrix model includes:

[0025] Capture a frame of audio and calculate the energy value of this frame of audio in the first frequency band;

[0026] Normalize the audio energy value;

[0027] The normalized audio energy value is classified into the corresponding division interval, and the corresponding division interval is set to 1;

[0028] Repeat the above steps to process the complete audio frame by frame, and finally establish a panic behavior audio feature recognition matrix model.

[0029] Furthermore, the panic behavior audio feature recognition matrix model is expressed as:

[0030]

[0031] Among them, C1, C2, C3, C4, C5, C6, C7, and C8 are the division intervals of audio energy.

[0032] Furthermore, C1, C3 and C5 are the division intervals for audio of panic behavior but unclear gender distinction, C2 is the division interval for female panic behavior audio, C4 is the division interval for male panic behavior audio, C6 and C8 are the division intervals for audio of indistinguishable behavior, and C7 is the division interval for normal communication behavior audio.

[0033] Furthermore, the behavior type and gender in the audio are determined based on the type of partition interval in which the audio energy in the audio feature recognition matrix model is mainly concentrated.

[0034] An electronic device, comprising:

[0035] one or more processors;

[0036] Memory;

[0037] One or more programs stored in a memory, wherein the one or more programs include instructions for executing any of the above-mentioned methods for recognizing audio features of screaming behavior of panicked pedestrians.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. The present invention takes into account the unclear content of pedestrians' shouting under the influence of panic emotions, focuses on screaming behavior, analyzes the audio features of the first frequency band that can effectively distinguish panic behavior from normal communication behavior, obtains the audio energy distribution and divides the audio energy by intervals, and establishes a panic behavior audio feature recognition matrix model to determine the behavior type and gender in the audio. This method makes the determination of panic pedestrian screaming behavior robust and adaptable to the environment, and can more effectively determine whether pedestrians are panicking.

[0040] 2. The present invention divides the energy value of the video audio into intervals by analyzing the audio energy of the screaming behavior of panic pedestrians of different genders, drawing a normal distribution fitting diagram, and calculating the audio energy mean and energy standard deviation, and then establishes a panic behavior audio feature recognition matrix model, which can effectively identify panic behavior while distinguishing the gender in the audio. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a process flow chart of the present invention;

[0042] Figure 2 This is the audio spectrum analysis diagram of normal male communication behavior;

[0043] Figure 3 This is an audio spectrum analysis diagram of normal female communication behavior;

[0044] Figure 4 This is the normal distribution fitting diagram of normal AC audio energy;

[0045] Figure 5 The audio spectrum energy normalization and error bar statistics for normal communication behavior;

[0046] Figure 6 Audio spectrum analysis diagram of female panic screaming behavior;

[0047] Figure 7 Audio spectrum analysis diagram of male panic screaming behavior;

[0048] Figure 8 The audio spectrum energy normalization and error bar statistics of panic screaming behavior;

[0049] Fig. 9 The energy normal distribution fitting diagram of the audio features of male panic screaming behavior and female panic screaming behavior. DETAILED DESCRIPTION

[0050] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0051] Example

[0052] This embodiment discloses a method for recognizing audio features of screaming behavior of panicked pedestrians. Figure 1 As shown, the method includes:

[0053] S1, extracts the audio part from the video;

[0054] S2, intercepting the audio features of the first frequency band in the audio, and calculating and obtaining the audio energy distribution;

[0055] S3, dividing the audio energy into intervals and establishing a panic behavior audio feature recognition matrix model;

[0056] S4, determining the behavior type and gender in the audio according to the audio feature recognition matrix model.

[0057] The specific implementation process of the method is as follows:

[0058] First, based on the audio data of normal communication behavior, the audio data features are analyzed.

[0059] The data set of normal communication behavior in this embodiment adopts a 200-hour Chinese Mandarin speech data set selected based on the complete data set, wherein the number of participants is 600, the sampling frequency is 16KHz 16bit, and 50 groups of data are randomly selected, including 30 males and 20 females.

[0060] Each frame of audio signal of normal communication behavior in the data set is extracted and Fourier transformed to obtain the waveform and spectrum of the audio signal. The frequency of the audio signal of normal communication behavior is mainly distributed below 6000Hz. The audio characteristics of normal communication behavior are analyzed in segments, the average energy of each frequency range is calculated, and the effective information of the spectral characteristics of the audio signal is extracted. The energy here represents the average length of the sound signal after Fourier transformation in each frequency range. It comes from the collected data and is a dimensionless value. The segmented calculation formula is shown as follows:

[0061]

[0062] In the formula, f internal is the length of each frequency segment, f min f is the minimum frequency of the collected audio, max The maximum frequency of the collected audio, n real is the number of divided frequency bands, f σ is the lower limit of the collection frequency. According to the statistical results of the collected information and the rationality of the data analysis, f max =6000Hz, f min =0Hz, n real =8.

[0063] The energy values ​​are summed in the 8 frequency bands of interest and normalized as shown in the following formula:

[0064]

[0065] Among them, e′ i Indicates the average energy proportion of each frequency band, e i Represents the average energy of each frequency band, e σ Indicates the lower limit of the average energy ratio, where e σ =0.

[0066] Finally, we get the statistical results of the average energy proportion of each frequency band in normal male and female communication behaviors. Figure 2 and Figure 3 As shown, Figure 2This is the audio spectrum analysis diagram of normal male communication behavior. Figure 3 This is an audio spectrum analysis diagram of normal female communication behavior.

[0067] Furthermore, if Figure 4 Based on the statistical results, a histogram is drawn and a normal density function is fitted, taking the first segment of frequency distribution as an example.

[0068] Figure 4 In the above equation, σ is the standard deviation, which is obtained through the confidence interval.

[0069] A confidence interval is an estimated interval of a population parameter constructed from a sample statistic. In statistics, a confidence interval of a probability sample is an interval estimate of a population parameter of the sample. The confidence interval shows the degree to which the true value of the parameter has a certain probability of falling around the measurement result. It gives the degree of credibility of the measured value of the measured parameter, that is, "a probability". The probability value of one standard deviation (1σ) is 68.3%, the probability value of two standard deviations (2σ) is 95.5%, and the probability value of three standard deviations (3σ) is 99.7%.

[0070] Figure 4 In , double standard deviation is selected, i.e., the confidence interval with a probability value of 95.5%. The mean and standard deviation obtained from the normal distribution fitting diagram of the audio energy of normal communication behavior are shown in Table 1.

[0071] Table 1 Mean and standard deviation of normal communication behavior audio

[0072] Serial number Frequency band Range(Hz) Energy Mean σ 1. <![CDATA[f s1 ]]> 0-750 0.405734 0.114096 2. <![CDATA[f s2 ]]> 750-1500 0.200814 0.070349 3. <![CDATA[f s3 ]]> 1500-2250 0.096148 0.047655 4. <![CDATA[f s4 ]]> 2250-3000 0.068588 0.030714 5. <![CDATA[f s5 ]]> 3000-3750 0.054678 0.025836 6. <![CDATA[f s6 ]]> 3750-4500 0.061152 0.037849 7. <![CDATA[f s7 ]]> 4500-5250 0.060240 0.046054 8. <![CDATA[f s8 ]]> 5250-6000 0.052646 0.039515

[0073] The energy normalization and errorbar statistics of the eight frequency bands of the audio spectrum of normal AC behavior are plotted using the above method, such as Figure 5 shown.

[0074] In the figure, f s6 (3750-4500Hz), f s7 (4500-5250Hz) and f s8 The lower limits of the error bars (5250-6000Hz) are all lower than 0. Based on the characteristic that the average energy ratio value is not lower than 0, the lower limits of the error bars that are lower than 0 are changed to 0.

[0075] The audio features of panic behavior are mainly reflected in the screaming characteristics of pedestrians. For this reason, the discussion begins with screaming behavior. The audio dataset of panic behavior is selected from the dataset Dataset-AOB: Urban Sound Event Classification. The dataset Dataset-AOB is an audio dataset for urban sound event classification collected and manually edited for a master's thesis using convolutional neural networks. The dataset contains 10 audio events: sirens, children playing, dogs barking, engines, footsteps, glass breaking, gunshots, subway trains, rain, and screams. The panic behavior of the crowd is often accompanied by screams. Screams in the Dataset-AOB dataset are randomly selected for analysis, and the statistical results of the average energy proportion of each frequency band of the audio of male panic behavior and female panic behavior are obtained. The audio spectrum analysis diagram of panic behavior is shown below. Figure 6 and Figure 7 As shown, Figure 6 This is the audio spectrum analysis diagram of female panic screaming behavior. Figure 7 Audio spectrum analysis of male panic screaming behavior.

[0076] The mean and standard deviation obtained from the normal distribution fitting diagram of the audio energy of panic behavior are shown in Table 2.

[0077] Table 2 Mean and standard deviation of panic behavior audio

[0078] Serial number Frequency band Range(Hz) Energy Mean σ 1. <![CDATA[f s1 ]]> 0-750 0.085062 0.060544 2. <![CDATA[f s2 ]]> 750-1500 0.404742 0.149134 3. <![CDATA[f s3 ]]> 1500-2250 0.149954 0.083362 4. <![CDATA[f s4 ]]> 2250-3000 0.114022 0.0895983 5. <![CDATA[f s5 ]]> 3000-3750 0.06373 0.0350749 6. <![CDATA[f s6 ]]> 3750-4500 0.07059 0.0387502 7. <![CDATA[f s7 ]]> 4500-5250 0.055618 0.0411158 8. <![CDATA[f s8 ]]> 5250-6000 0.050986 0.0303498

[0079] According to the audio spectrum energy normalization and error bar statistical analysis method of normal communication behavior, the audio spectrum energy normalization and error bar statistical diagram of panic behavior are obtained as follows: Figure 8 shown.

[0080] Figure 8 Medium s1 (0-750Hz), f s3 (1500-2250Hz), f s4 (2250-3000Hz), f s5 (3000-3750Hz), f s6 (3750-4500Hz), f s7 (4500-5250Hz) and f s8 The lower limits of the error bars (5250-6000Hz) are all lower than 0. Based on the characteristic that the average energy ratio value is not lower than 0, the lower limits of the error bars that are lower than 0 are changed to 0.

[0081] Based on the above data, it is concluded that the audio feature analysis of normal communication behavior and the audio feature analysis of panic behavior are similar in f s1 (0-750Hz) there is a clear difference.

[0082] Therefore, in the audio feature recognition method for panicked pedestrian screaming behavior of this embodiment, the first frequency band is the 0-750 Hz frequency band.

[0083] In practical applications, in addition to effectively identifying panic behavior audio in audio, it is also important to effectively identify the gender of panic behavior audio. Therefore, we randomly selected male and female panic behavior audio in the dataset and analyzed the panic behavior of the male and female audio. s1 (0-750Hz), the mean and standard deviation of the normal distribution fitting diagram of the audio energy of male panic behavior and female panic behavior are shown in Table 3, and the energy normal distribution fitting diagram of the audio characteristics of male panic behavior and female panic behavior is shown in Fig. 9 shown.

[0084] Table 3 Mean and standard deviation of audio data of male / female panic behavior

[0085] Serial number gender Frequency band Range(Hz) Energy Mean σ 1. male <![CDATA[f s1 ]]> 0-750 0.124179 0.047869 2. female <![CDATA[f s1 ]]> 0-750 0.031043 0.022134

[0086] according to Fig. 9 Compared with Table 3, it can be concluded that the audio energy distribution of panic behaviors of males and females is different.

[0087] Furthermore, the energy proportion is divided into eight intervals: 0-0.009, 0.009-0.053, 0.053-0.076, 0.076-0.172, 0.172-0.178, 0.178-0.206, 0.206-0.634, and 0.634-1.

[0088] Furthermore, a panic behavior audio feature recognition matrix model is established, and the process includes:

[0089] Capture a frame of audio and calculate the energy value of this frame of audio in the first frequency band;

[0090] Normalize the audio energy value;

[0091] The normalized audio energy value is classified into the corresponding division interval, and the corresponding division interval is set to 1;

[0092] Repeat the above steps, process the complete audio frame by frame, and finally establish the panic behavior audio feature recognition matrix model.

[0093] Furthermore, the expression of the audio feature recognition matrix model is:

[0094]

[0095] Among them, C1, C2, C3, C4, C5, C6, C7, and C8 are the division intervals of audio energy.

[0096] Furthermore, C1, C3 and C5 are the division intervals for audio of panic behavior but unclear gender distinction, C2 is the division interval for female panic behavior audio, C4 is the division interval for male panic behavior audio, C6 and C8 are the division intervals for audio of indistinguishable behavior, and C7 is the division interval for normal communication behavior audio.

[0097] Furthermore, the behavior type and gender in the audio are determined based on the type of partition interval in which the audio energy in the audio feature recognition matrix model is mainly concentrated.

[0098] Taking a certain place as an example, using the surveillance video of the place, the audio feature recognition model of panic pedestrian screaming behavior is verified through real panic event video.

[0099] Extract the audio part of a video from a certain place and extract f s1 (0-750Hz) frequency band audio features. By normalizing the audio spectrum energy, calculate f s1 The audio energy distribution of the frequency band, the energy value after normalization should belong to the interval (0,1). According to the eight energy distribution intervals divided, the f obtained by normalization calculation is s1 The energy results in the frequency band are placed in the corresponding energy interval. For example, at the moment t = 1s in the video, if f s1 The audio energy of the frequency band is 0.033 after normalization, and the C2 interval at this moment is set to 1, indicating that female panic screaming behavior is detected in the audio at this time. According to the above method, the complete audio is normalized and data is placed in the corresponding interval to obtain the calculation result of the panic behavior audio feature recognition matrix model, which is expressed as [0,1,0,0,0,0,0,0], that is, C2 panic behavior is recognized in the audio, and it is considered that female panic screaming behavior appears in the video.

[0100] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0101] In another embodiment, an electronic device is provided, characterized in that it includes one or more processors, a memory, and one or more programs stored in the memory, wherein the one or more programs include instructions for executing the method for identifying audio features of panic pedestrian screaming behavior as described above.

[0102] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for identifying audio features of screaming behavior of panicked pedestrians, characterized in that: The method comprises: Extract the audio portion from the video; Intercepting the audio features of the first frequency band in the audio, and calculating and obtaining the audio energy distribution; Divide the audio energy into intervals and establish a matrix model for panic behavior audio feature recognition; Determine the behavior type and gender in the audio based on the audio feature recognition matrix model.

2. The method for recognizing audio features of panicked pedestrian screaming behavior according to claim 1, characterized in that: The first frequency band is a 0-750 Hz frequency band.

3. The method for recognizing audio features of panicked pedestrian screaming behavior according to claim 1, characterized in that: The process of intercepting the audio features of the first frequency band in the audio and calculating the audio energy distribution includes: Perform Fourier transform on the audio data to obtain the waveform and spectrum of the audio signal; Extracting audio features of a first frequency band from the frequency spectrum of the audio signal; The audio energy of the first frequency band is normalized to calculate the audio energy distribution.

4. The method for recognizing audio features of panicked pedestrian screaming behavior according to claim 1, characterized in that: The process of dividing the audio energy into intervals includes: Get the panic pedestrian screaming behavior audio dataset; Randomly select male panic behavior audio and female panic behavior audio from the dataset and analyze the energy distribution of the first frequency band audio; Draw the energy normal distribution fitting diagram of male panic behavior audio and female panic behavior audio, and calculate the energy mean and energy standard deviation; The audio energy is divided into intervals based on the normal distribution fitting diagram of audio energy of different genders, energy mean and energy standard deviation.

5. The method for recognizing audio features of panicked pedestrian screaming behavior according to claim 4, characterized in that: The audio energy division intervals include: [0, 0.009], [0.009, 0.053], [0.053, 0.076], [0.076, 0.172], [0.172, 0.178], [0.178, 0.206], [0.206, 0.634] and [0.634, 1].

6. The method for recognizing audio features of panicked pedestrian screaming behavior according to claim 1, characterized in that: The process of establishing the panic behavior audio feature recognition matrix model includes: Capture a frame of audio and calculate the energy value of this frame of audio in the first frequency band; Normalize the audio energy value; The normalized audio energy value is classified into the corresponding division interval, and the corresponding division interval is set to 1; Repeat the above steps, process the complete audio frame by frame, and finally establish the panic behavior audio feature recognition matrix model.

7. The method for recognizing audio features of panicked pedestrian screaming behavior according to claim 1, characterized in that: The expression of the panic behavior audio feature recognition matrix model is: Among them, C1, C2, C3, C4, C5, C6, C7, and C8 are the division intervals of audio energy.

8. The method for recognizing audio features of panicked pedestrian screaming behavior according to claim 7, characterized in that: C1, C3 and C5 are the division intervals for audio with panic behavior but ambiguous gender distinction, C2 is the division interval for female panic behavior audio, C4 is the division interval for male panic behavior audio, C6 and C8 are the division intervals for audio with indistinguishable behavior, and C7 is the division interval for normal communication behavior audio.

9. The method for recognizing audio features of panicked pedestrian screaming behavior according to claim 7, characterized in that: According to the type of partition interval where the audio energy in the audio feature recognition matrix model is mainly concentrated, the behavior type and gender in the audio are judged.

10. An electronic device, characterized in that: include: one or more processors; Memory; One or more programs stored in a memory, the one or more programs comprising instructions for executing the method for recognizing audio features of screaming behavior of panicked pedestrians as claimed in any one of claims 1-9.

Images