Pet emotion recognition method and system based on sound analysis

Abstract

The invention relates to the technical field of intelligent voice, in particular to a pet emotion recognition method and system based on sound analysis, and the method comprises the following steps: extracting a dominant frequency value, a frequency spectrum gravity center and an energy concentration degree through audio collection equipment, and respectively mapping the dominant frequency value, the frequency spectrum gravity center and the energy concentration degree into three-dimensional coordinate axes to generate time sequence emotion state points; and calculating a sound intensity change rate by utilizing Gaussian kernel density estimation, judging an emotion activation source through a tension response function to generate a response graph, executing gradient calculation and non-maximum suppression to generate a label intensity graph, and generating a composite emotion expression result by combining a path turning angle and the graph. According to the method, emotional coordinates are mapped through a main frequency value, a frequency spectrum gravity center and an energy concentration degree, sound intensity changes are estimated through Gaussian kernel density, disturbance and space are correlated through a tension response function, a focused emotional interval is suppressed through a non-maximum value, path turning is analyzed through dynamic time warping, emotional evolution is described through a label map, and judgment continuity and consistency are improved through multi-dimensional feature fusion.

Description

Technical Field

[0001] The present invention relates to the field of intelligent voice technology, and in particular to a pet emotion recognition method and system based on call analysis. Background Art

[0002] The field of intelligent speech technology involves speech signal processing, voiceprint feature extraction, and pattern recognition technologies. Its core is to analyze speech content and recognize emotional states through acoustic parameter analysis. It is primarily used in human-computer interaction systems, biometric recognition systems, and affective computing systems. Traditional pet emotion recognition methods involve collecting animal vocalizations and analyzing their acoustic characteristic parameters to determine emotional states. Specifically addressing the technical challenge of determining the correlation between vocalization characteristics and emotional states in pet emotion recognition, traditional methods typically employ voiceprint feature analysis, including fundamental frequency change rate, resonance peak distribution, and time-frequency characteristic parameter extraction, combined with preset threshold comparisons or simple classification models to achieve emotional classification.

[0003] Traditional methods rely on threshold comparisons of static acoustic parameters such as the fundamental frequency change rate and resonance peak distribution. The preset classification model is difficult to adapt to the dynamic fluctuations of acoustic features in the time dimension. The isolated analysis of feature parameters leads to one-sided judgment of the correlation between emotional states. The threshold setting relies on empirical data and is not robust enough to individual differences and environmental interference. Simple classification models cannot capture the spatiotemporal distribution characteristics of emotional activation sources. The nonlinear relationship between the energy concentration of the resonance peak frequency band and the intensity of the emotion is ignored. The extraction of time-frequency characteristic parameters lacks a multi-scale fusion mechanism. The emotional category classification is easily interfered by single feature outliers, and the recognition results are misjudged or missed in complex acoustic scenarios. Summary of the Invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a pet emotion recognition method and system based on call analysis.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a pet emotion recognition method based on call analysis, comprising the following steps: S1: Obtain pet sound signals through audio acquisition equipment, extract the main frequency value, spectrum center of gravity, and energy concentration for each frame of the signal, map the main frequency value to the x-axis component, map the spectrum center of gravity to the y-axis component, and map the energy concentration to the oscillation period ratio to the z-axis component, and generate a three-dimensional emotional state coordinate set of the time series; S2: Based on the three-dimensional emotional state coordinate set, Gaussian kernel density estimation is used to calculate the sound intensity change rate within the neighborhood radius r, and a tension response function is called to perform a product operation on the main frequency disturbance rate and the inverse of the distance. When the cumulative value exceeds a threshold, the emotional activation source is marked and a tension response distribution map is generated; S3: Based on the tension response distribution map, performing radial gradient calculation on the emotion activation source, using a non-maximum suppression algorithm to filter the gradient rising area, and generating a label intensity map; S4: Based on the three-dimensional emotional state coordinate set, a dynamic time warping algorithm is used to calculate the path turning angle. When the angle exceeds 25 degrees, the path is divided into turning segments, and the composite emotional expression result is generated in combination with the label strength map.

[0006] As a further solution of the present invention, when the ratio of the energy concentration to the oscillation period is mapped to the z-axis component, a dimensionless process is performed, the energy concentration is divided by 100 to convert it into a scalar value, and the oscillation period is multiplied by the sampling frequency to convert it into a dimensionless period parameter; The threshold is optimized in the embodiment by the gradient descent method, and the optimization objective function is: ,in, Represents the labeled emotion intensity value of the i-th sample, i is the sample index number and , n is the total number of training set samples, and the annotation value interval is [0,10], Representative model input features for the i-th sample The predicted output value of Contains the x, y, z axis components and timestamp information of the three-dimensional emotional state coordinate set, is the model parameter vector, including the bandwidth parameter of the Gaussian kernel density estimate and the slope coefficient of the tension response function , is the L2 regularization coefficient and its value is 0.01, Indicates the sum of the loss values of n training samples.

[0007] As a further solution of the present invention, the steps of obtaining the three-dimensional emotional state coordinate set are specifically as follows: S101: Acquire a pet sound signal through an audio acquisition device, frame the signal using a Hanning window function, perform a fast Fourier transform on each frame of the signal, detect the frequency component with the maximum amplitude in the frequency domain, record the value as the main frequency, and generate a framed main frequency sequence corresponding to the time axis; S102: Based on the spectrum distribution of the framed main frequency sequence, a weighted average of the amplitudes of multiple frequency components and their corresponding frequencies is calculated to obtain the spectrum centroid of each frame signal, and a median filtering process is performed on the spectrum centroids of five adjacent frames using a sliding window method to generate a noise-reduced spectrum centroid sequence; S103: Call the time domain waveform of the framed main frequency sequence, calculate the signal zero-crossing rate as the oscillation period parameter, extract the energy proportion of the multi-frame signal in the 0-8kHz frequency band as the energy concentration, perform a ratio operation on the energy concentration and the oscillation period parameter, map the main frequency value, the spectrum center of gravity sequence, and the energy oscillation ratio into the x, y, and z axis components of the three-dimensional coordinate system, and generate a three-dimensional emotional state coordinate set.

[0008] As a further solution of the present invention, the step of obtaining the tension response distribution map is specifically as follows: S201: Based on the three-dimensional emotional state coordinate set, a probability density estimation model is constructed using a Gaussian kernel function. The standard deviation of the sound intensity values within a neighborhood radius centered on each coordinate point is calculated. The ratio of the standard deviation to the time interval is used as the sound intensity change rate, and a sound intensity change rate field covering all coordinate points is generated. S202: Calling the x-axis component of the three-dimensional emotional state coordinate set, calculating the absolute value of the difference between the main frequency values at adjacent time points, obtaining the main frequency disturbance rate, performing a scalar multiplication operation on the main frequency disturbance rate and the inverse of the Euclidean distance from the corresponding coordinate point to the sound source point, and generating a tension response coefficient matrix; S203: Setting an activation threshold in the sound intensity change rate field, performing a sliding window accumulation operation on the tension response coefficient matrix, marking the center point of the region as the emotion activation source when the accumulated value in the window exceeds the threshold, constructing a spatial distribution surface through interpolation, and generating a tension response distribution map.

[0009] As a further solution of the present invention, the sliding window adopts a 3×3×2 space-time cube structure, and the time dimension covers the current frame and each frame of data before and after it.

[0010] As a further solution of the present invention, the steps of obtaining the tag intensity map are specifically as follows: S301: Based on the tension response distribution map, a polar coordinate system is established with the emotional activation source as the center point, sampling points are set at equal angles along eight directions, and the absolute value of the difference in response values between adjacent sampling points in multiple directions is calculated to generate a radial gradient field; S302: Calling the gradient data of the radial gradient field, dividing the field into sixteen equally divided sectors in the circumferential direction, detecting the maximum coordinates of the gradient modulus in each sector, and retaining the local maximum points by comparing the gradient modulus values of three adjacent sectors to generate a candidate gradient extreme value set; S303: Perform spatial cluster analysis on the candidate gradient extreme value set, calculate the Euclidean distance between extreme value points, merge adjacent points with a spacing smaller than the neighborhood radius, calculate the mean gradient intensity within the merged area, and generate a label intensity map.

[0011] As a further solution of the present invention, when sampling in the polar coordinate system, the radial sampling interval is linearly related to the neighborhood radius r. ,in The polar diameter.

[0012] As a further solution of the present invention, the steps of obtaining the composite emotion expression result are specifically as follows: S401: Based on the three-dimensional emotional state coordinate set, a dynamic time warping algorithm is used to align the time series, the direction vector differences between adjacent coordinate points are calculated, the vector coordinates are converted into radians using an inverse tangent function, and the turning angles of consecutive coordinate points are stored in an array to generate a path turning angle sequence; S402: Calling the path turning angle sequence, establishing an angle change rate calculation formula, setting 25 degrees as the turning determination threshold, scanning the angle array to mark the index of the exceeding limit point, performing interval merging operations on consecutive indexes, recording the starting and ending positions of the turning segments, and generating a path turning segment set; S403: Extract the timestamp information of the path turning segment set, map the intensity matrix of the label intensity map to the corresponding moment, use the regional weighted algorithm to calculate the intensity mean of the turning segment coverage area, perform matrix multiplication operation on the spatial distribution feature and the time turning feature, and generate a composite emotion expression result.

[0013] As a further solution of the present invention, the time dimension weight coefficient 0.6 corresponds to the duration feature of the path turning segment, the space dimension weight coefficient 0.4 corresponds to the spatial distribution discreteness of the label intensity map, and the feature vector length variance is calculated by Calculate, where is the Euclidean length of the kth eigenvector, is the mean of the lengths of all eigenvectors, is the total number of eigenvectors.

[0014] A pet emotion recognition system based on vocalization analysis is provided, wherein the pet emotion recognition system based on vocalization analysis is used to implement the above-mentioned pet emotion recognition method based on vocalization analysis. The system comprises: A three-dimensional feature mapping module is used to obtain pet sound signal frames through an audio acquisition device, extract the main frequency value, spectral center of gravity, and energy concentration of each frame signal, map the main frequency value to the x-axis component, map the spectral center of gravity to the y-axis component, and map the energy concentration to the oscillation period ratio to the z-axis component, generate a three-dimensional emotional state coordinate set of the time series, and transmit the three-dimensional emotional state coordinate set to the tension response generation module; a tension response generation module configured to use Gaussian kernel density estimation to calculate the rate of change of sound intensity within a neighborhood radius r for the three-dimensional emotional state coordinate set, perform a product operation on the main frequency disturbance rate and the inverse of the distance using a tension response function, mark an emotional activation source when the cumulative value exceeds a set threshold, generate a tension response distribution map, and transmit the emotional activation source and the tension response distribution map to a gradient intensity analysis module; a gradient intensity analysis module, configured to perform radial gradient calculation on the emotion activation source using the tension response distribution map, filter the gradient rising region using a non-maximum suppression algorithm, generate a label intensity map, and transmit the label intensity map to the temporal emotion integration module; A temporal emotion integration module is configured to calculate the path turning angle of the three-dimensional emotional state coordinate set using a dynamic time warping algorithm, divide the path turning segment when the angle exceeds 25 degrees, perform emotion type matching on the path turning segment in combination with the label intensity map, and generate a composite emotion expression result; The emotion type matching is implemented using a convolutional neural network, and the network input layer includes a concatenated tensor of a time transition feature matrix and a spatial intensity matrix.

[0015] Compared with the prior art, the advantages and positive effects of the present invention are: In the present invention, by mapping the main frequency value, spectrum center of gravity, energy concentration and oscillation period ratio into three-dimensional emotion coordinates, a dynamic emotion representation of the time series is constructed, and the neighborhood sound intensity change rate is quantified by combining Gaussian kernel density estimation. The tension response function is used to dynamically capture the correlation characteristics between the main frequency disturbance and the spatial distance, so as to achieve accurate positioning of the emotion activation source. The non-maximum suppression algorithm screens the gradient rising area and effectively focuses on the core emotion expression interval. The dynamic time warping algorithm analyzes the path turning angle. The label intensity map is combined to analyze the time evolution law of complex emotions. The multi-dimensional feature fusion and dynamic response mechanism enhance the continuity of emotion state discrimination, improve the recognition sensitivity and result consistency in complex emotion interaction scenarios, and solve the limitations of traditional methods in analyzing the correlation of time series features and the superposition state of complex emotions. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the workflow of the present invention; Figure 2 A flowchart of the steps for obtaining a three-dimensional emotional state coordinate set according to the present invention; Figure 3 Flowchart of the steps for obtaining the tension response distribution diagram of the present invention; Figure 4 Flowchart of the steps for obtaining the label intensity map of the present invention; Figure 5 This is a flow chart of the steps for obtaining the composite emotion expression results of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0018] In the description of the present invention, it should be understood that the terms "length," "width," "up," "down," "front," "back," "left," "right," "vertical," "horizontal," "top," "bottom," "inside," "outside," and the like, indicating positions or relationships, are based on the positions or relationships shown in the accompanying drawings and are intended only to facilitate the description of the present invention and simplify the description. They do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting the present invention. Furthermore, in the description of the present invention, "plurality" means two or more, unless otherwise expressly and specifically defined.

[0019] Example 1: Please refer to Figure 1 The present invention provides a technical solution: a pet emotion recognition method based on call analysis, comprising the following steps: S1: Obtain pet sound signals through audio acquisition equipment, extract the main frequency value, spectrum center of gravity, and energy concentration for each frame of the signal, map the main frequency value to the x-axis component, map the spectrum center of gravity to the y-axis component, and map the energy concentration to the oscillation period ratio to the z-axis component, and generate a three-dimensional emotional state coordinate set of the time series; S2: Based on the three-dimensional emotional state coordinate set, Gaussian kernel density estimation is used to calculate the sound intensity change rate within the neighborhood radius r. The tension response function is called to multiply the main frequency disturbance rate and the inverse of the distance. When the cumulative value exceeds the threshold, the emotional activation source is marked and a tension response distribution map is generated. S3: Based on the tension response distribution map, radial gradient calculation is performed on the emotional activation source, and the non-maximum suppression algorithm is used to filter the gradient rising area to generate the label intensity map; S4: Based on the three-dimensional emotional state coordinate set, the dynamic time warping algorithm is used to calculate the path turning angle. When the angle exceeds 25 degrees, the path is divided into turning segments, and the composite emotional expression result is generated by combining the label intensity map.

[0020] When the ratio of energy concentration to oscillation period is mapped to the z-axis component, dimensionless processing is performed: the energy concentration is divided by 100 to convert it into a scalar value, and the oscillation period is multiplied by the sampling frequency to convert it into a dimensionless period parameter; The threshold is optimized in the embodiment by the gradient descent method, and the optimization objective function is: ,in, Represents the labeled emotion intensity value of the i-th sample, i is the sample index number and , n is the total number of training set samples, and the annotation value interval is [0,10], Representative model input features for the i-th sample The predicted output value of Contains the x, y, z axis components and timestamp information of the three-dimensional emotional state coordinate set, is the model parameter vector, including the bandwidth parameter of the Gaussian kernel density estimate and the slope coefficient of the tension response function , is the L2 regularization coefficient and its value is 0.01, Indicates the sum of the loss values of n training samples.

[0021] See also Figure 2 , the specific steps for obtaining the three-dimensional emotional state coordinate set are: S101: Acquire a pet sound signal through an audio acquisition device, frame the signal using a Hanning window function, perform a fast Fourier transform on each frame of the signal, detect the frequency component with the maximum amplitude in the frequency domain, record the value as the main frequency, and generate a framed main frequency sequence corresponding to the time axis; The cat meow signal with a sampling rate of 44.1kHz and a duration of 2 seconds is obtained through the audio acquisition device. The signal is framed using a window function. In the specific implementation, the frame length is set to 1024 sampling points, the frame shift is 512 sampling points, and the Hanning window function is selected. The window function is constructed by the cosine function, and the two ends smoothly transition to zero. Its calculation formula in the time domain is: ,in The window length is 1024, is the sampling point index, from 0 to 1023, and each frame signal is multiplied by the Hanning window function. A 1024-point fast Fourier transform is performed on each processed frame signal to convert it from the time domain to the frequency domain. The amplitude of each frequency point is obtained after the transformation. In the frequency domain range of 0Hz to 22.05kHz, the frequency component with the largest amplitude is detected. For example, for the 10th frame signal, after Fourier transform, the frequency component at 750Hz is detected to have the highest amplitude of 0.98, so the main frequency value of the 10th frame is recorded as 750Hz. This operation is repeated for all frames to generate a frame main frequency sequence corresponding to the time axis (or frame number). The sequence is expressed as a series of discrete main frequency values, such as the starting part of the sequence is [720Hz, 725Hz, 735Hz, 740Hz, 750Hz]. Finally, a frame main frequency sequence corresponding to the time axis is generated.

[0022] S102: Based on the spectrum distribution of the framed main frequency sequence, a weighted average of the amplitudes of multiple frequency components and their corresponding frequencies is calculated to obtain the spectrum centroid of each frame signal. A median filter is performed on the spectrum centroids of five adjacent frames using a sliding window method to generate a noise-reduced spectrum centroid sequence. Based on the frequency domain data of each frame obtained in the above steps and the frame main frequency sequence, the complete spectrum distribution of each frame signal is called, and the weighted average of the amplitudes of multiple frequency components and their corresponding frequencies is calculated. This calculation does not use a preset model, but directly operates on the data. Specifically, the frequency value of each frequency point is multiplied by its corresponding amplitude, and then all these products are added to obtain the total weighted sum, which is then divided by the sum of all amplitudes. For example, for the 10th frame signal, there are three significant frequency peaks in its spectrum, with frequencies and amplitudes of (740Hz, 0.85), (750Hz, 0.98), and (765Hz, 0.80), respectively. The calculation process of its spectrum center of gravity is: Hz, which is the spectral centroid of the 10th frame. After performing this calculation on all frames, the obtained spectral centroid sequence is subjected to median filtering using the sliding window method. The sliding window length here is fixed at 5 frames. The setting is based on the fact that experiments have found that windows smaller than 3 frames cannot effectively filter out short-term random noise, while windows larger than 5 frames will be over-smoothed, resulting in loss of emotional transition details. Taking a 5-frame window as an example, if the spectral centroid values of five consecutive frames are [745.1, 748.5, 751.3, 749.2, 753.8], then these five values are sorted as [745.1, 748.5, 749.2, 751.3, 753.8], and the middle value 749.2 is taken as the filtered value of the center frame of the window (i.e., the 3rd frame). The window is then slid across the entire sequence, and calculations are performed frame by frame to finally generate a denoised spectral centroid sequence.

[0023] S103: Calling the time domain waveform of the framed main frequency sequence, calculating the signal zero-crossing rate as the oscillation period parameter, extracting the energy proportion of the multi-frame signal in the 0-8kHz frequency band as the energy concentration, performing a ratio operation on the energy concentration and the oscillation period parameter, mapping the main frequency value, the spectrum center of gravity sequence, and the energy oscillation ratio to the x-axis, y-axis, and z-axis components of the three-dimensional coordinate system, and generating a three-dimensional emotional state coordinate set; The time domain waveform of the framed main frequency sequence generated by S101 is called, and the signal zero-crossing rate is calculated for the analysis window composed of every 15 frames of data. The frame interval of 15 is set according to the 1:3 ratio based on the median filter window length 5 of S102, which is used to capture the oscillation characteristics of the sound on a longer time scale and match the smoothing process of the spectrum center of gravity. When calculating, the number of times the signal amplitude changes from positive to negative or from negative to positive in all the sampling points contained in the 15 frames is counted, and this count value is used as the oscillation period parameter of the center frame of the analysis window. For example, with the 8th frame as the center, the signals of frames 1 to 15 are analyzed, and a total of Sampling points, if the number of times the signal passes through the zero value is 600, then the oscillation period parameter of the 8th frame is 600. At the same time, the energy proportion of these 15 frames of signals in the 0-8kHz frequency band is extracted as the energy concentration. The selection of this frequency band is based on the statistical analysis of a large number of cat vocal databases. It is found that when expressing clear emotions, more than 90% of their sound energy is concentrated in this frequency band. When calculating, first calculate the sum of the energy of the 15 frames of signals in 0-8kHz, and then calculate the sum of the energy of the full frequency band. The energy concentration is obtained by dividing the two. If the 0-8kHz energy sum is 1.44 and the full-band energy sum is 1.60, the energy concentration is ,, then, perform a ratio operation on the energy concentration and the oscillation period parameter to obtain , which is the energy oscillation ratio. Finally, the main frequency value obtained in S101, the denoised spectrum center sequence obtained in S102, and the energy oscillation ratio calculated here are respectively mapped to the x-axis, y-axis, and z-axis components of the three-dimensional coordinate system. For example, for the 8th frame, its main frequency value is 735 Hz, the filtered spectrum center of gravity is 742.6 Hz, and the energy oscillation ratio is 0.0015, then its three-dimensional coordinates are (735, 742.6, 0.0015). This operation is performed for all frames to generate a three-dimensional emotional state coordinate set.

[0024] See also Figure 3 , the steps for obtaining the tension response distribution map are as follows: S201: Based on the three-dimensional emotional state coordinate set, a probability density estimation model is constructed using a Gaussian kernel function. The standard deviation of the sound intensity values within a neighborhood radius centered on each coordinate point is calculated. The ratio of the standard deviation to the time interval is used as the sound intensity change rate, and a sound intensity change rate field covering all coordinate points is generated. Based on the three-dimensional emotional state coordinate set generated in the previous steps, each point in this set represents the sound characteristics of a time frame and is additionally associated with a sound intensity value. This sound intensity value is obtained by calculating the root mean square energy of the signal in the corresponding time frame. First, a probability density estimation model is constructed using a Gaussian kernel function. The standard deviation of the sound intensity within the neighborhood of each coordinate point is calculated. The neighborhood radius is set based on the average distance between the reference coordinate points. If the average nearest neighbor distance of the coordinate point in space is 12 units, the neighborhood radius is set to 1. 5 times the distance, that is, 18 units. For the coordinate point P8 (735, 742.6, 0.0015), with it as the center, find 5 adjacent coordinate points within a radius of 18 units. The corresponding sound intensity values are [0.75, 0.78, 0.81, 0.79, 0.76], and the standard deviation of this group of sound intensity values is calculated, and the result is 0.0228. Then, the ratio of the standard deviation to the time interval is used as the sound intensity change rate. The time interval is determined by the sampling rate and frame shift of the audio signal, that is, seconds, then the sound intensity change rate at point P8 is , repeat this calculation for all coordinate points in the three-dimensional emotional state coordinate set to generate a sound intensity change rate field covering all coordinate points.

[0025] S202: Calling the x-axis component of the three-dimensional emotional state coordinate set, calculating the absolute value of the difference between the main frequency values at adjacent time points, obtaining the main frequency disturbance rate, performing a scalar multiplication operation on the main frequency disturbance rate and the inverse of the Euclidean distance from the corresponding coordinate point to the sound source point, and generating a tension response coefficient matrix; Call the x-axis component of the three-dimensional emotional state coordinate set, that is, the frame main frequency sequence, and calculate the absolute difference of the main frequency values of adjacent time points to obtain the main frequency disturbance rate. For example, the main frequency value of the 7th frame is 740Hz, and the main frequency value of the 8th frame is 735Hz. Then the main frequency disturbance rate of the 8th frame is Hz, then, perform a scalar multiplication operation on the main frequency disturbance rate and the inverse of the Euclidean distance from the corresponding coordinate point to the sound source point. The sound source point is set to the point with the highest sound intensity value in the coordinate set. By traversing the sound intensity values associated with all points, the maximum sound intensity is determined to be 0.95, and its corresponding coordinates are P_source(850,855.2,0.0012). For the coordinate point P8(735,742.6,0.0015), the Euclidean distance from it to the sound source point P_source is calculated as , take its reciprocal to get , multiply the main frequency disturbance rate 5Hz by the reciprocal value, , which is the tension response coefficient value of point P8. Perform the same calculation on all coordinate points to generate the tension response coefficient matrix.

[0026] S203: setting an activation threshold in the sound intensity change rate field, performing a sliding window accumulation operation on the tension response coefficient matrix, marking the center point of the region as the emotion activation source when the accumulated value in the window exceeds the threshold, constructing a spatial distribution surface through interpolation, and generating a tension response distribution map; An activation threshold is set in the sound intensity change rate field generated by S201. The setting of this threshold refers to the results of the audio analysis of 500 pet calls with known emotions (such as relaxation and alertness). The distribution of their sound intensity change rates is statistically analyzed. It is found that when the pet's emotion changes from relaxation to alertness or excitement, the sound intensity change rate generally exceeds 1.8. Therefore, the activation threshold is set to 1.8, and a sliding window accumulation operation is performed on the tension response coefficient matrix generated by S202. The sliding window adopts a 3×3×2 space-time cube structure. The spatial dimension covers the 3x3 neighborhood around the coordinate point, and the time dimension covers the data of the current frame and the frame before and after it. To ensure the capture of dynamic changes, when the cumulative sum of all tension response coefficient values in the window exceeds the preset threshold, the center point of the marked area is the emotion activation source. The setting of this cumulative threshold is also based on historical data analysis and is set to 0.65. When the cumulative value in the window, for example 0.72, exceeds 0.65, the center point of the window is marked. After completing the scanning and marking of all points, a continuous spatial distribution surface is constructed through interpolation, specifically radial basis function interpolation. This surface smoothly connects all tension response coefficient values and highlights the high-value area where the emotion activation source is located, finally generating a tension response distribution map.

[0027] The sliding window adopts a 3×3×2 space-time cube structure, and the time dimension covers the current frame and every frame before and after it.

[0028] See also Figure 4 , the steps for obtaining the label intensity map are as follows: S301: Based on the tension response distribution map, a polar coordinate system is established with the emotional activation source as the center point, sampling points are set at equal angles along eight directions, and the absolute value of the difference in response values between adjacent sampling points in multiple directions is calculated to generate a radial gradient field; Based on the tension response distribution map obtained in the previous steps, a polar coordinate system is established with the emotional activation source identified in the map as the center point. If multiple activation sources are identified, this operation is performed independently for each activation source. With one of the activation sources P_active (x, y, z) as the origin, sampling rays are set at equal angles along eight directions, namely 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°. On each ray, The sampling interval is is the neighborhood radius, according to the setting of S201, units, so the radial sampling interval Units, starting from the origin, along each direction, with a step length of 3.6, outward sampling, record the response value of each sampling point on the tension response distribution diagram, for example, in the 45° direction, the obtained sampling point response value sequence is [V1, V2, V3, ...], then calculate the absolute value of the difference between the response values of adjacent sampling points in multiple directions, that is, , Etc., to obtain an approximate gradient at each orientation, and this operation is performed for all eight orientations to generate a radial gradient field that describes how drastically the response value changes in space.

[0029] S302: Calling the gradient data of the radial gradient field, dividing the circumferential direction into sixteen equally divided sectors, detecting the maximum coordinates of the gradient modulus in each sector, and retaining the local maximum points by comparing the gradient modulus values of three adjacent sectors to generate a candidate gradient extreme value set; The gradient data of the radial gradient field generated in the above steps are called and divided into sixteen equally divided sectors in the circumferential direction, that is, within the range of 0° to 360°. Each sector covers an angular range of 22.5°. The maximum value of the gradient modulus in each sector and its coordinate position in the sector are detected and recorded. For example, in the first sector (0°-22.5°), the maximum value of all the calculated gradient modulus values is 5.8, and its position is ( =7.2, =15°), then record this maximum point. After completing the detection of all sixteen sectors, retain the local maximum point by comparing the maximum values of the gradient modulus of the three adjacent sectors. The specific operation is, for the maximum value of each sector, check whether it is greater than the maximum value of its left and right adjacent sectors. For example, if the maximum values of the 8th, 9th, and 10th sectors are [4.5, 6.2, 5.1] respectively, then the maximum value 6.2 of the 9th sector is retained because it is greater than 4.5 and 5.1. If it is [4.5, 5.1, 6.2], then the value 5.1 of the 9th sector is not retained. By traversing all sectors through this method, the set of points screened out is the candidate gradient extreme value set.

[0030] S303: Perform spatial cluster analysis on the candidate gradient extreme value set, calculate the Euclidean distance between extreme value points, merge adjacent points whose distance is less than the neighborhood radius, calculate the mean gradient intensity in the merged area, and generate a label intensity map; Perform spatial clustering analysis on the candidate gradient extreme value set generated in the previous step, calculate the Euclidean distance between each pair of extreme value points in the set, and merge the extreme value points whose spacing is less than the neighborhood radius. (i.e. 18 units) of adjacent points, the merging operation adopts the agglomerative clustering method. First, each extreme point is regarded as an independent cluster, and then the two closest clusters are iteratively merged until the minimum distance between all clusters is greater than 18 units. For example, if there are three extreme points P1, P2, and P3, and the distances between them are d(P1, P2) = 15, d(P1, P3) = 25, and d(P2, P3) = 22, then P1 and P2 are first merged to form a new cluster {P1, P2}, and then the distance between the new cluster and P3 is calculated. If it is greater than 18, the merging is stopped, and the mean gradient intensity in each merged area is counted. For cluster {P1, P2}, if the gradient intensity of P1 is 8.1 and the gradient intensity of P2 is 8.5, then the mean gradient intensity of the area is , and assign the mean value to the geometric center point of the merged area, and finally obtain a set of spatially separated areas each with an intensity value, which is the label intensity map.

[0031] When sampling in polar coordinates, the radial sampling interval is linearly related to the neighborhood radius r ,in The polar diameter.

[0032] See also Figure 5 ,The specific steps for obtaining the composite emotion expression result are: S401: Based on the three-dimensional emotional state coordinate set, a dynamic time warping algorithm is used to align the time series, calculate the direction vector differences between adjacent coordinate points, convert the vector coordinates into radians using the inverse tangent function, store the turning angles of consecutive coordinate points in an array, and generate a path turning angle sequence; Based on the three-dimensional emotional state coordinate set generated in S103, this set is time series data. In order to calculate the direction vector between adjacent coordinate points, the dynamic time warping algorithm is first used to align the time series. However, in this specific embodiment, since the coordinate points are generated at equal time intervals (frame shifts), the time series itself is aligned, so the subsequent calculation is directly performed to calculate the direction vector difference between each two consecutive coordinate points. Specifically, for three consecutive points P(t-1), P(t), and P(t+1) in the sequence, the vector V1 (from P(t-1) to P(t)) and the vector V2 (from P(t) to P(t+1)) are calculated, and then the inverse tangent function atan2(y,x) is used to convert these points. The coordinates of the two vectors are converted to radians, and the angle between them is calculated. For example, P(t-1)=(735,742.6,0.0015), P(t)=(730,738.1,0.0018), P(t+1)=(738,735.5,0.0017), vector V1=(-5,-4.5,0.0003), vector V2=(8,-2.6,-0.0001), and the angle between these two three-dimensional vectors is calculated to be approximately 2.1 radians (approximately 120 degrees). This angle is used as the turning angle at time t, and the steering angles calculated for all consecutive coordinate points are stored in an array to generate a path turning angle sequence.

[0033] S402: Calling the path turning angle sequence, establishing an angle change rate calculation formula, setting 25 degrees as the turning threshold, scanning the angle array to mark the index of the exceeding limit point, performing interval merging operations on consecutive indexes, recording the starting and ending positions of the turning segments, and generating a path turning segment set; Call the path turning angle sequence generated in the previous steps to establish the calculation of the angle change rate. Here, it is simplified to directly use the size of the turning angle itself for judgment, and set 25 degrees (about 0.436 radians) as the turning judgment threshold. The setting of this threshold is based on an experiment involving 1,000 labeled samples of pet behavior experts. The experiment shows that when the emotional state changes significantly, the turning angle of its motion trajectory in the three-dimensional emotional space has a 95% probability of exceeding 25 degrees. Turns below this angle are generally considered to be a steady drift of the emotional state. Scan the angle array and mark all those exceeding The turning point index exceeds 25 degrees. For example, if the turning angle sequence (unit: degree) is [15,22,45,50,30,18,35], then indexes 2,3,4,6 (corresponding to values 45,50,30,35) will be marked, and then the interval merging operation is performed on the consecutive out-of-limit indexes. Indexes 2,3,4 are continuous, so they are merged into a turning segment with the starting position at index 2 and the ending position at index 4. Index 6 is discontinuous and forms a single-point turning segment. Through this method, the starting and ending positions of all turning segments are recorded, and finally a path turning segment set is generated.

[0034] S403: Extracting the timestamp information of the path turning segment set, mapping the intensity matrix of the label intensity map to the corresponding moment, calculating the intensity mean of the turning segment coverage area using a regional weighted algorithm, performing matrix multiplication operation on the spatial distribution feature and the temporal turning feature to generate a composite emotion expression result; Extract the timestamp information of the path turning segment set. For example, identify a turning segment lasting from the 25th frame to the 35th frame, and then map the intensity matrix corresponding to the label intensity map generated in S303 at these time points (25th frame to 35th frame). Use the regional weighted algorithm to calculate the intensity mean of the area covered by the turning segment, that is, average all the label intensity values in the label intensity map corresponding to each of the 11 frames to obtain a mean representing the spatial feature intensity in the time period, set as S_mean, perform matrix multiplication operation on the spatial distribution feature and the time turning feature. Specifically, construct a feature vector, which contains the duration of the path turning segment (11 frames) and the spatial distribution dispersion of the label intensity map. The spatial distribution dispersion is calculated by the formula Complete, in this formula, represents the variance of the eigenvector length, i.e. the spatial distribution discreteness, is the total number of feature vectors in all label intensity maps within this time period (frames 25 to 35), is the Euclidean length of the kth eigenvector (i.e., the distance from the center of the activation source to the label), This is it The arithmetic mean of the lengths of the feature vectors is used to quantify the degree of discreteness of the label distribution in space by calculating the variance of the feature vector lengths. The larger the variance, the more discrete the distribution. Assume that during the 25th to 35th frame, a total of feature vectors, whose lengths are , calculate its mean , for example, the mean is 25.4, and then the variance formula is used to calculate , which is the spatial distribution discreteness. Finally, the duration feature (11 frames) and the spatial distribution discreteness (3.1) are combined according to the weights. The time dimension weight coefficient is 0.6, and the spatial dimension weight coefficient is 0.4. These two weight coefficients are obtained by grid search optimization on the labeled data set to maximize the accuracy of emotion classification. The final composite emotion expression score is , this value is a composite emotion expression result. The benefit of the formula is that by introducing the variance of the feature vector length , which can quantify the spatial diffusion or contraction of emotions after activation, rather than simply the average intensity. This combines temporal dynamics (path turning) with spatial morphology (distribution dispersion), providing richer multimodal emotional information. The result of 7.84 indicates that this path turning event, which lasted for a long time (high weight contribution) and had a relatively concentrated spatial distribution (low variance contribution), contained a complex emotional expression of moderate intensity.

[0035] The time dimension weight coefficient of 0.6 corresponds to the duration feature of the path turning segment, the space dimension weight coefficient of 0.4 corresponds to the spatial distribution discreteness of the label intensity map, and the feature vector length variance is calculated by Calculate, where is the Euclidean length of the kth eigenvector, is the mean of the lengths of all eigenvectors, is the total number of eigenvectors.

[0036] A pet emotion recognition system based on vocalization analysis is used to implement the above-mentioned pet emotion recognition method based on vocalization analysis. The system includes: A three-dimensional feature mapping module is used to obtain pet sound signal frames through an audio acquisition device, extract the main frequency value, spectral center of gravity, and energy concentration of each frame signal, map the main frequency value to the x-axis component, map the spectral center of gravity to the y-axis component, and map the energy concentration to the oscillation period ratio to the z-axis component. This generates a three-dimensional emotional state coordinate set of the time series, and transmits the three-dimensional emotional state coordinate set to the tension response generation module; The tension response generation module is used to call Gaussian kernel density estimation to calculate the rate of change of sound intensity within the neighborhood radius r of the three-dimensional emotional state coordinate set. The tension response function is used to perform a product operation on the main frequency disturbance rate and the inverse of the distance. When the cumulative value exceeds the set threshold, the emotional activation source is marked and a tension response distribution map is generated. The emotional activation source and the tension response distribution map are passed to the gradient intensity analysis module. The gradient intensity analysis module is used to perform radial gradient calculation on the emotion activation source through the tension response distribution map, use the non-maximum suppression algorithm to filter the gradient rising area, generate the label intensity map, and pass the label intensity map to the temporal emotion integration module; The temporal emotion integration module is used to calculate the path turning angle of the three-dimensional emotional state coordinate set using a dynamic time warping algorithm. When the angle exceeds 25 degrees, the path is divided into turning segments. The path turning segments are then matched with the label intensity map to generate a composite emotion expression result. Emotion type matching is implemented using a convolutional neural network, and the network input layer contains a concatenated tensor of the temporal transition feature matrix and the spatial intensity matrix.

[0037] The above are merely preferred embodiments of the present invention and do not limit the present invention in any other form. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A pet emotion recognition method based on call analysis, characterized in that: The following steps are involved: S1: Obtain pet sound signals through audio acquisition equipment, extract the main frequency value, spectrum center of gravity, and energy concentration for each frame of the signal, map the main frequency value to the x-axis component, map the spectrum center of gravity to the y-axis component, and map the energy concentration to the oscillation period ratio to the z-axis component, and generate a three-dimensional emotional state coordinate set of the time series; S2: Based on the three-dimensional emotional state coordinate set, Gaussian kernel density estimation is used to calculate the sound intensity change rate within the neighborhood radius r, and a tension response function is called to perform a product operation on the main frequency disturbance rate and the inverse of the distance. When the cumulative value exceeds a threshold, the emotional activation source is marked and a tension response distribution map is generated; S3: Based on the tension response distribution map, performing radial gradient calculation on the emotion activation source, using a non-maximum suppression algorithm to filter the gradient rising area, and generating a label intensity map; S4: Based on the three-dimensional emotional state coordinate set, a dynamic time warping algorithm is used to calculate the path turning angle. When the angle exceeds 25 degrees, the path is divided into turning segments, and the composite emotional expression result is generated in combination with the label strength map.

2. The pet emotion recognition method based on call analysis according to claim 1, characterized in that: When the ratio of the energy concentration to the oscillation period is mapped to the z-axis component, a dimensionless process is performed, the energy concentration is divided by 100 to convert it into a scalar value, and the oscillation period is multiplied by the sampling frequency to convert it into a dimensionless period parameter; The threshold is optimized in the embodiment by the gradient descent method, and the optimization objective function is: ,in, Represents the labeled emotion intensity value of the i-th sample, i is the sample index number and , n is the total number of training set samples, and the annotation value interval is [0,10], Representative model input features for the i-th sample The predicted output value of Contains the x, y, z axis components and timestamp information of the three-dimensional emotional state coordinate set, is the model parameter vector, including the bandwidth parameter of the Gaussian kernel density estimate and the slope coefficient of the tension response function , is the L2 regularization coefficient and its value is 0.01, Indicates the sum of the loss values of n training samples.

3. The pet emotion recognition method based on call analysis according to claim 2, characterized in that: The steps for obtaining the three-dimensional emotional state coordinate set are specifically as follows: S101: Acquire a pet sound signal through an audio acquisition device, frame the signal using a Hanning window function, perform a fast Fourier transform on each frame of the signal, detect the frequency component with the maximum amplitude in the frequency domain, record the value as the main frequency, and generate a framed main frequency sequence corresponding to the time axis; S102: Based on the spectrum distribution of the framed main frequency sequence, a weighted average of the amplitudes of multiple frequency components and their corresponding frequencies is calculated to obtain the spectrum centroid of each frame signal, and a median filtering process is performed on the spectrum centroids of five adjacent frames using a sliding window method to generate a noise-reduced spectrum centroid sequence; S103: Call the time domain waveform of the framed main frequency sequence, calculate the signal zero-crossing rate as the oscillation period parameter, extract the energy proportion of the multi-frame signal in the 0-8kHz frequency band as the energy concentration, perform a ratio operation on the energy concentration and the oscillation period parameter, map the main frequency value, the spectrum center of gravity sequence, and the energy oscillation ratio into the x, y, and z axis components of the three-dimensional coordinate system, and generate a three-dimensional emotional state coordinate set.

4. The pet emotion recognition method based on call analysis according to claim 3 is characterized in that: The steps for obtaining the tension response distribution map are specifically as follows: S201: Based on the three-dimensional emotional state coordinate set, a probability density estimation model is constructed using a Gaussian kernel function. The standard deviation of the sound intensity values within a neighborhood radius centered on each coordinate point is calculated. The ratio of the standard deviation to the time interval is used as the sound intensity change rate, and a sound intensity change rate field covering all coordinate points is generated. S202: Calling the x-axis component of the three-dimensional emotional state coordinate set, calculating the absolute value of the difference between the main frequency values at adjacent time points, obtaining the main frequency disturbance rate, performing a scalar multiplication operation on the main frequency disturbance rate and the inverse of the Euclidean distance from the corresponding coordinate point to the sound source point, and generating a tension response coefficient matrix; S203: Setting an activation threshold in the sound intensity change rate field, performing a sliding window accumulation operation on the tension response coefficient matrix, marking the center point of the region as the emotion activation source when the accumulated value in the window exceeds the threshold, constructing a spatial distribution surface through interpolation, and generating a tension response distribution map.

5. The pet emotion recognition method based on call analysis according to claim 4 is characterized in that: The sliding window adopts a 3×3×2 space-time cube structure, and the time dimension covers the current frame and each frame of data before and after it.

6. The pet emotion recognition method based on call analysis according to claim 5, characterized in that: The steps for obtaining the label intensity map are specifically as follows: S301: Based on the tension response distribution map, a polar coordinate system is established with the emotional activation source as the center point, sampling points are set at equal angles along eight directions, and the absolute value of the difference in response values between adjacent sampling points in multiple directions is calculated to generate a radial gradient field; S302: Calling the gradient data of the radial gradient field, dividing the field into sixteen equally divided sectors in the circumferential direction, detecting the maximum coordinates of the gradient modulus in each sector, and retaining the local maximum points by comparing the gradient modulus values of three adjacent sectors to generate a candidate gradient extreme value set; S303: Perform spatial cluster analysis on the candidate gradient extreme value set, calculate the Euclidean distance between extreme value points, merge adjacent points with a spacing smaller than the neighborhood radius, calculate the mean gradient intensity within the merged area, and generate a label intensity map.

7. The pet emotion recognition method based on call analysis according to claim 6, characterized in that: When sampling in the polar coordinate system, the radial sampling interval is linearly related to the neighborhood radius r. ,in The polar diameter.

8. The pet emotion recognition method based on call analysis according to claim 7, characterized in that: The steps for obtaining the composite emotion expression result are specifically as follows: S401: Based on the three-dimensional emotional state coordinate set, a dynamic time warping algorithm is used to align the time series, the direction vector differences between adjacent coordinate points are calculated, the vector coordinates are converted into radians using an inverse tangent function, and the turning angles of consecutive coordinate points are stored in an array to generate a path turning angle sequence; S402: Calling the path turning angle sequence, establishing an angle change rate calculation formula, setting 25 degrees as the turning determination threshold, scanning the angle array to mark the index of the exceeding limit point, performing interval merging operations on consecutive indexes, recording the starting and ending positions of the turning segments, and generating a path turning segment set; S403: Extract the timestamp information of the path turning segment set, map the intensity matrix of the label intensity map to the corresponding moment, use the regional weighted algorithm to calculate the intensity mean of the turning segment coverage area, perform matrix multiplication operation on the spatial distribution feature and the time turning feature, and generate a composite emotion expression result.

9. The pet emotion recognition method based on call analysis according to claim 8, characterized in that: The time dimension weight coefficient of 0.6 corresponds to the duration feature of the path turning segment, the space dimension weight coefficient of 0.4 corresponds to the spatial distribution discreteness of the label intensity map, and the feature vector length variance is calculated by Calculate, where is the Euclidean length of the kth eigenvector, is the mean of the lengths of all eigenvectors, is the total number of eigenvectors.

10. A pet emotion recognition system based on vocalization analysis, characterized in that: The system is used to implement the pet emotion recognition method based on call analysis according to any one of claims 1 to 9, and the system includes: A three-dimensional feature mapping module is used to obtain pet sound signal frames through an audio acquisition device, extract the main frequency value, spectral center of gravity, and energy concentration of each frame signal, map the main frequency value to the x-axis component, map the spectral center of gravity to the y-axis component, and map the energy concentration to the oscillation period ratio to the z-axis component, generate a three-dimensional emotional state coordinate set of the time series, and transmit the three-dimensional emotional state coordinate set to the tension response generation module; a tension response generation module configured to use Gaussian kernel density estimation to calculate the rate of change of sound intensity within a neighborhood radius r for the three-dimensional emotional state coordinate set, perform a product operation on the main frequency disturbance rate and the inverse of the distance using a tension response function, mark an emotional activation source when the cumulative value exceeds a set threshold, generate a tension response distribution map, and transmit the emotional activation source and the tension response distribution map to a gradient intensity analysis module; a gradient intensity analysis module, configured to perform radial gradient calculation on the emotion activation source using the tension response distribution map, filter the gradient rising region using a non-maximum suppression algorithm, generate a label intensity map, and transmit the label intensity map to the temporal emotion integration module; A temporal emotion integration module is configured to calculate the path turning angle of the three-dimensional emotional state coordinate set using a dynamic time warping algorithm, divide the path turning segment when the angle exceeds 25 degrees, perform emotion type matching on the path turning segment in combination with the label intensity map, and generate a composite emotion expression result; The emotion type matching is implemented using a convolutional neural network, and the network input layer includes a concatenated tensor of a time transition feature matrix and a spatial intensity matrix.

Images