Recording data storage method and device and storage medium
By applying noise reduction processing to the recordings and using a hierarchical framework for extracting emotional factors, the problems of difficult classification and storage of recording data and recognition of fluctuating emotions were solved, thereby improving the accuracy of recording emotion recognition and storage.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-24
- Publication Date
- 2026-03-13
AI Technical Summary
Existing methods for storing audio recordings struggle to categorize and store data based on the emotions of multiple recording subjects, and they also have difficulty accurately identifying fluctuating emotions during recording emotion recognition, leading to difficulties in retrieval and analysis.
By denoising the recordings, extracting text and speech data, determining reference emotion factors based on the speech data, and using a hierarchical emotion factor extraction framework, including position weighting and smoothing, the recording emotion types are identified, classified, and stored.
It improves the accuracy of recorded emotion recognition and storage, can more accurately capture language post-preferences, reduces overfitting interference, balances accuracy and efficiency, and is suitable for emotion recognition and storage.
Smart Images
Figure CN121662087A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of audio recording and storage technology, and in particular to a method, apparatus and storage medium for recording data storage. Background Technology
[0002] With the rapid development of information technology, the Internet of Things, and artificial intelligence, audio, as a key information carrier, has permeated all aspects of social production and life through its recording, storage, and application. From traditional judicial evidence collection and customer service center call recordings to emerging smart home voice interaction, autonomous driving environmental sound recognition, and massive online meetings and content creation, the scale of audio recording data is exploding at an unprecedented rate.
[0003] Currently, most methods for saving audio recordings neglect to categorize and store the emotions of multiple recording subjects, instead storing them as a whole. This makes it difficult to retrieve recordings with specific emotions, hinders the rapid analysis of customer emotional fluctuations, and makes it difficult to optimize service strategies accordingly. Furthermore, most methods struggle to accurately identify fluctuating emotions during emotion recognition. When multiple emotions appear in a sentence, they cannot accurately identify the emotional state of that sentence, reducing the accuracy of emotion analysis and storage.
[0004] Therefore, the present invention discloses a method, apparatus and storage medium for recording data storage, which is used to solve the above-mentioned technical problems. Summary of the Invention
[0005] This application provides a method, apparatus, and storage medium for storing audio recording data, which solves the technical problems in the prior art of classifying and storing recordings according to the emotions of multiple recording subjects, and of accurately identifying fluctuating emotions during the emotion recognition of recordings.
[0006] To achieve the above objectives, this application adopts the following technical solution: Firstly, a method for saving audio recording data is provided, including: The recording is noise-reduced to obtain the text and speech data of the noise-reduced recording; the text data includes the intervals between the recorded text; the speech data includes the pitch, speech rate and volume of the recorded text. Reference sentiment factors for each recorded text are determined based on the speech data, and basic sentiment factors for each sentence are determined based on the reference sentiment factors. The basic sentiment factors of each sentence are corrected by analyzing the text data of each sentence to determine the characteristic sentiment factors of each sentence. Extract the emotion type from this recording and store the recording based on the characteristic emotion factors and emotion type.
[0007] In conjunction with the first aspect mentioned above, one possible implementation involves noise reduction processing of the recording, including: The recorded signal is subjected to spectral analysis to extract noise frequency band features. The noise type is then classified using support vector machine and deep neural network, and noise type labels are output. The frequency band features include high-frequency noise and low-frequency noise. The noise type labels include environmental noise, mechanical noise, and mixed human voice noise. The filter parameters are adjusted based on noise type labels and frequency band characteristics. The adjustment of filter parameters is based on adaptive filtering algorithms, including LMS and RLS algorithms, to track noise characteristics in real time and optimize filter performance. The recording signal is divided into high-frequency, mid-frequency, and low-frequency bands, and the recording signal of each band is denoised using adaptive filter parameters that match the noise type of the corresponding frequency band.
[0008] In conjunction with the first aspect mentioned above, one possible implementation involves acquiring the text and speech data of the denoised recording, including: Recordings are identified by timbre, and sentences with the same timbre are marked as the same recognition object; text recognition technology is used to obtain the recorded text of each recognition object and the spacing between each character in the recorded text; among them, speech recognition technology includes Speech-to-Text API and Azure Speech Services. The tone, speed, and volume of each character in the recorded text of each identified object are obtained through speech analysis tools, including Praat and Audacity.
[0009] In conjunction with the first aspect mentioned above, one possible implementation involves determining reference emotion factors for each recorded text based on speech data, including: Extract the pitch, speech rate, volume, and reference sentiment factors of each recorded text from the reference recordings in the reference database; wherein, the reference database includes the pitch, speech rate, and volume of several recorded texts, as well as the reference sentiment factors set by experts based on the pitch, speech rate, and volume of the recorded texts. The pitch, speech rate, volume, and reference emotion factors of each recorded text are integrated into several sets of training data and test data. The training data is used to train the artificial intelligence model, and the test data is used to test the trained artificial intelligence model. The artificial intelligence model is adjusted according to the test results. Finally, an emotion recognition model is obtained with the pitch, speech rate, and volume of the recorded text as input and the reference emotion factors of each recorded text as output. The artificial intelligence model includes a BP neural network model and an RBF neural network model.
[0010] In conjunction with the first aspect mentioned above, in one possible implementation, the basic sentiment factors for each statement are determined based on reference sentiment factors, including: Sequentially extract sentences from the recording, extract the position number (WH) of each recorded text within the current sentence; extract the reference sentiment factor for each text in the current sentence. The text is marked as the target text in order from back to front according to the position number WH; where i represents the emotion type number, and the value of i is in the range of [1,n], and n is the maximum value of the emotion type number; j represents the magnitude of the emotion value; the emotion types include neutral emotion, happiness, anger, sadness and fear; When the target text has a different emotion type from both of its two adjacent texts, determine whether the emotion types of the two adjacent texts are the same. If yes, change the emotion type of the target text to the emotion type of the two adjacent texts and set the base emotion factor of the target text to 0. If no, do nothing. Determine whether only one emotion type exists in the extracted statement; if yes, determine the basic emotion factor of the current statement based on formula (1). No, retrieve the number of characters corresponding to each emotion type in the current sentence. And based on the calculation formula (2), the preliminary emotion factors corresponding to various emotion types in the current sentence are determined. The largest initial emotional factor Multiply Obtain the basic sentiment factor of the current statement ;in, The error adjustment coefficient is set based on experience, and The value range of is [1,2]; The calculation formula (1) is: ; Where ZS is the total number of characters in the current statement; The amplitude adjustment coefficient is set based on experience, and The value range is [1, 1.71828]; The calculation formula (2) is: .
[0011] In conjunction with the first aspect mentioned above, one possible implementation involves determining the characteristic sentiment factors of each statement by modifying the basic sentiment factors using the text data of each statement, including: Extracting the basic sentiment factors of the current statement The corresponding emotion type is identified by using an emotion text database to identify text belonging to that emotion type in the current sentence, and the proportion of that text to the total text is obtained (BL). Extract the intervals between each recorded text in the current sentence, and mark the intervals from beginning to end as follows: The rate of change of the interval between two adjacent intervals is determined sequentially based on the calculation formula (3). For several of the aforementioned interval change rates The average rate of change, PGB, is obtained by calculating the average value. Based on proportional BL, average rate of change PGB, and basic sentiment factor The characteristic emotional factors of the current statement are determined by calculation formula (4). ; The calculation formula (3) is: ; The calculation formula (4) is: .
[0012] In conjunction with the first aspect mentioned above, one possible implementation involves extracting the emotion type from the recording, including: Extracting characteristic sentiment factors from each statement and characteristic emotional factors The corresponding emotion type is used to label each statement according to its emotion type. .
[0013] In conjunction with the first aspect above, one possible implementation involves storing the recording based on characteristic emotion factors and emotion type, including: Recordings of the same subject were categorized by emotion type. Categorized storage based on emotion type Set the text of each sentence to the corresponding type color, and according to the mood type. Corresponding characteristic emotion factors Determine the RGB value of the color of the current statement type; The recordings are stored as a whole, based on emotion type. Set the time bar and text for each statement in the recording to the corresponding type color, and adjust them according to the emotion type. Corresponding characteristic emotion factors Determine the RGB value of the current statement's type color; where, in the type color, when the emotion type... When the emotion is neutral, the color is off-white; when the emotion type... When feeling happy, the color is yellow; when the emotion type is yellow... When angry, the type color is red; when the emotion type is red... When feeling sad, the color is blue; when the emotion type is... When experiencing fear, the type color is black.
[0014] In a second aspect, a recording data storage device is provided, comprising: a communication unit and a processing unit; The communication unit is used to perform noise reduction processing on the recording and obtain the text data and voice data of the noise-reduced recording; wherein, the text data includes the recording text and the interval between the recording text; the voice data includes the pitch, speech rate and volume of the recording text; The processing unit is used to: determine reference emotion factors for each recorded text based on speech data; determine basic emotion factors for each sentence based on reference emotion factors; determine characteristic emotion factors for each sentence by correcting the basic emotion factors using text data of each sentence; extract the emotion type of the current recording; and store the recording based on the characteristic emotion factors and emotion type.
[0015] Thirdly, this application provides a storage medium storing instructions that, when executed on a recording data storage device, cause the recording data storage device to perform the method described in the first aspect and any possible implementation thereof.
[0016] This application provides a method, apparatus, and storage medium for storing audio recording data, which has the following advantages: 1. This invention addresses the technical problems of existing technologies in classifying and storing recordings based on the emotions of multiple recording subjects, and in accurately identifying fluctuating emotions during emotion recognition. This invention improves the accuracy of emotion recognition and storage in recordings.
[0017] 2. This invention constructs a hierarchical emotion factor extraction framework: the positional weight mechanism accurately captures language post-preferences, improving the emotion resolution of single sentences; the smoothing process enhances local consistency and reduces overfitting interference; single / multiple emotion branches and exponential ratio adjustment provide an adaptive calculation path, balancing accuracy and efficiency; this invention can promote the evolution of emotion recognition and storage of recordings towards a more accurate direction.
[0018] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description
[0019] Figure 1 This is a schematic diagram illustrating the steps of a method for saving audio recording data provided in an embodiment of this application; Figure 2 This application provides a schematic diagram illustrating the steps for obtaining basic emotion factors in its embodiments. Figure 3 This is a schematic diagram of the structure of a recording data storage device provided in an embodiment of this application. Detailed Implementation
[0020] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.
[0021] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.
[0022] like Figure 1 As shown in the embodiment of this application, a method for saving audio recording data includes: S1. Perform noise reduction processing on the recording to obtain the text data and speech data of the noise-reduced recording; wherein, the text data includes the intervals between the recorded text; the speech data includes the pitch, speech rate and volume of the recorded text. S2. Determine the reference sentiment factor for each recorded text based on the speech data, and determine the basic sentiment factor for each sentence based on the reference sentiment factor. S3. The basic emotional factors are corrected by modifying the text data of each sentence to determine the characteristic emotional factors of each sentence. S4. Extract the emotion type of this recording and store the recording based on the characteristic emotion factors and emotion type.
[0023] In one possible implementation of this application embodiment, the above-mentioned S1 can be implemented by the following S101 and S102, which are described in detail below: S101. Perform noise reduction processing on the recording, including: The recorded signal is subjected to spectral analysis to extract noise frequency band features. The noise type is then classified using support vector machine and deep neural network, and noise type labels are output. The frequency band features include high-frequency noise and low-frequency noise. The noise type labels include environmental noise, mechanical noise, and mixed human voice noise. The filter parameters are adjusted based on noise type labels and frequency band characteristics. The adjustment of filter parameters is based on adaptive filtering algorithms, including LMS and RLS algorithms, to track noise characteristics in real time and optimize filter performance. The recording signal is divided into high-frequency, mid-frequency, and low-frequency bands, and the recording signal of each band is denoised using adaptive filter parameters that match the noise type of the corresponding frequency band.
[0024] It should be noted that when adjusting the filter parameters according to the noise type label, a high cutoff frequency is set for high-frequency noise and a low cutoff frequency is set for low-frequency noise; the high and low cutoff frequencies are obtained manually.
[0025] It should be noted that the specific steps for noise reduction processing of the recording are as follows: Fast Fourier Transform (FFT) spectrum analysis is performed on the recording signal to calculate the power spectral density of each frequency band and extract the frequency band features. The frequency band features include high-frequency noise and low-frequency noise. High-frequency noise is the frequency component whose power spectrum is significantly higher than the background mean in the range of 4kHz–8kHz, and low-frequency noise is the frequency component whose power spectrum is significantly higher than the background mean in the range of 20Hz–300Hz. The extracted frequency band features are input into a trained noise classification model to obtain noise type labels. The noise classification model can be a support vector machine (SVM, kernel function RBF, penalty parameter C=1.0) or a deep neural network (DNN, containing 3 fully connected layers with 128, 64, and 3 nodes respectively). The noise type labels include environmental noise, mechanical noise, and mixed human voice noise. The adaptive filter parameters are adjusted according to the noise type label and frequency band characteristics. Specifically, when the noise is low-frequency, the low-pass cutoff frequency is adjusted to below 300Hz; when the noise is high-frequency, the high-pass cutoff frequency is adjusted to above 4kHz. The filter uses the LMS algorithm (step size μ=0.01) and the RLS algorithm (forgetting factor λ=0.99) to update the weights in real time according to the input noise characteristics. The recording signal is divided into high frequency, mid frequency and low frequency bands; among them, low frequency band: 20Hz–300Hz, mid frequency band: 300Hz–4kHz, high frequency band: 4kHz–8kHz; Noise reduction processing is performed on the recording signals of each frequency band using adaptive filter parameters that match the noise type of the corresponding frequency band.
[0026] S102. Obtain the text and speech data of the noise-reduced recording, including: Recordings are identified by timbre, and sentences with the same timbre are marked as the same recognition object; text recognition technology is used to obtain the recorded text of each recognition object and the spacing between each character in the recorded text; among them, speech recognition technology includes Speech-to-Text API and Azure Speech Services. The tone, speed, and volume of each character in the recorded text of each identified object are obtained through speech analysis tools, including Praat and Audacity.
[0027] In one possible implementation of this application embodiment, the above-mentioned S2 can be implemented by the following S201 and S202, which are described in detail below: S201. Based on the speech data, determine the reference emotion factors for each recorded text, including: Extract the pitch, speech rate, volume, and reference sentiment factors of each recorded text from the reference recordings in the reference database; wherein, the reference database includes the pitch, speech rate, and volume of several recorded texts, as well as the reference sentiment factors set by experts based on the pitch, speech rate, and volume of the recorded texts. The pitch, speech rate, volume, and reference emotion factors of each recorded text are integrated into several sets of training data and test data. The training data is used to train the artificial intelligence model, and the test data is used to test the trained artificial intelligence model. The artificial intelligence model is adjusted according to the test results. Finally, an emotion recognition model is obtained with the pitch, speech rate, and volume of the recorded text as input and the reference emotion factors of each recorded text as output. The artificial intelligence model includes a BP neural network model and an RBF neural network model.
[0028] It should be noted that the reference sentiment factor This includes emotion type and factor size; where i represents the emotion type and j represents the emotion value; for example, refer to the emotion factor. for This indicates that the emotion type is happiness, and the reference emotion factor value corresponding to happiness is 2.2.
[0029] Specifically, the steps for testing the trained AI model using validation data and adjusting the AI model based on the validation results are as follows: The pitch, speech rate, and volume of each recorded text in the test data are input into the trained artificial intelligence model to obtain the corresponding reference emotion factor. The corresponding reference emotion factor is compared with the corresponding reference emotion factor in the test data. If the difference between the two is within a threshold (the threshold is obtained empirically), no parameter adjustment is required, and the next set of test data is tested. If it is not within the threshold, the corresponding parameters are adjusted until the output reference emotion factor of the corresponding test data is within the threshold, and the next set of test data is tested. When the number of test data in which the difference of the reference emotion factor obtained from all test data is within the threshold accounts for 90% or more of the total test data, an emotion recognition model is obtained with the pitch, speech rate, and volume of the recorded text as input and the reference emotion factor of each recorded text as output.
[0030] It should be noted that when comparing the corresponding reference sentiment factors with the corresponding reference sentiment factors in the test data, there may be reference sentiment factors that are not explicitly stated in the original text. If the values of 'i' and 'i' are different, it means that the types of emotions are different. In this case, the difference between the two values is set to infinity.
[0031] like Figure 2 As shown in S202, the basic sentiment factors for each statement are determined based on reference sentiment factors, including: Sequentially extract sentences from the recording, extract the position number (WH) of each recorded text within the current sentence; extract the reference sentiment factor for each text in the current sentence. The text is marked as the target text in order from back to front according to the position number WH; where i represents the emotion type number, and the value of i is in the range of [1,n], and n is the maximum value of the emotion type number; j represents the magnitude of the emotion value; the emotion types include neutral emotion, happiness, anger, sadness and fear; When the target text has a different emotion type from both of its two adjacent texts, determine whether the emotion types of the two adjacent texts are the same. If yes, change the emotion type of the target text to the emotion type of the two adjacent texts and set the base emotion factor of the target text to 0. If no, do nothing. Determine whether only one emotion type exists in the extracted statement; if yes, determine the basic emotion factor of the current statement based on formula (1). No, retrieve the number of characters corresponding to each emotion type in the current sentence. And based on the calculation formula (2), the preliminary emotion factors corresponding to various emotion types in the current sentence are determined. The largest initial emotional factor Multiply Obtain the basic sentiment factor of the current statement ;in, The error adjustment coefficient is set based on experience, and The value range of is [1,2]; The calculation formula (1) is: ; Where ZS is the total number of characters in the current statement; The amplitude adjustment coefficient is set based on experience, and The value range is [1, 1.71828]; The calculation formula (2) is: .
[0032] It is worth noting that this step first extracts sentences from the recording sequentially and assigns positional numbers WH to the text in each sentence, marking the target text from back to front. This shifts the semantic focus of the sentences towards the latter half, because human expression often follows the principle of "important information at the end." For example, in a transitional sentence like "Although it started calmly, it ended with anger," the latter part, "filled with anger," carries the dominant emotion, while in a question like "Is everything alright? No, it terrifies me," the tension of fear is amplified at the end. By referencing emotional factors... The extraction of WH values, combined with the processing order from back to front, ensures that the WH values of the later characters are relatively large, thus affecting the calculation in formulas (1) and (2). Weighting function This naturally amplifies the contribution of the latter half of the sentence, avoiding the common problem of the first half of the sentence interfering with the dominant emotion under uniform weighting, which is especially suitable for spoken language recording scenarios; in spoken language recording scenarios, sentences often unfold emotional arcs non-linearly. Then, when the emotional type of the target text is different from both of its two adjacent texts, the target text type is changed accordingly by judging whether the two adjacent texts are the same. This smoothing operation simulates the continuity and context dependence of human language, avoiding fragmented analysis errors caused by isolated emotional words (such as fear in the suddenly inserted "suddenly"). For example, in a mixed sentence such as "In a happy party, a sad memory flashes by, but happiness is quickly restored", this step can smooth the isolated "sadness" to the type of the surrounding "happiness", reducing false positive emotional fluctuations and thus improving the overall accuracy of emotional consistency extraction. At the same time, it retains computational efficiency because it only makes local adjustments to heterogeneous triples, rather than recalculating globally. This not only reduces computational complexity but also enhances the robustness of the system and makes it more tolerant of emotional fluctuation recordings or dialect variations. Finally, when determining the singularity of the emotional type of a statement, if there is only one emotion, the calculation formula (1) is directly used. This simplifies the process and faithfully reflects the position-weighted intensity of pure emotions, which is suitable for scenarios such as "a monologue filled with sadness" to ensure that the high weight of the latter part strengthens the dominant tone. If there are multiple emotions, the proportion of the number of words is introduced. index adjustment ,in The proportion of dominant emotions is converted into a multiplicative amplification factor, thereby highlighting the weight of the majority emotions and preventing the dilution of overall judgment by the minority emotions (such as temporary fear); finally, the largest preliminary emotion factor is selected. Multiply This further optimizes the decision boundary under multi-emotion competition. For example, in a statement that "starts with anger and ends with happiness," the latter part is happy. Although not high, the error adjustment coefficient Amplification can ensure basic emotional factors The value significantly improved the stability of emotion recognition.
[0033] Overall, this step constructs a hierarchical framework for extracting sentiment factors: the positional weighting mechanism accurately captures language post-positional preferences, improving the sentiment resolution of single sentences; the smoothing process enhances local consistency and reduces overfitting interference; and the single / multiple sentiment branches and exponential ratio adjustment provide an adaptive calculation path, balancing accuracy and efficiency. The introduction of coefficients adds practical flexibility, facilitating domain-specific fine-tuning. These innovations can drive the evolution of emotion recognition storage in audio recordings towards greater accuracy.
[0034] It should be noted that the position number WH is obtained by sorting the text from front to back according to the order in which the text appears; for example, in "What are you saying", the position number WH of "you" is 1, the position number of "in" is 2, the position number of "say" is 3, the position number of "what" is 4, and the position number of "what" is 5.
[0035] It should be noted that, It is a function whose value increases with WH, and its range is [0.5, 0, 999]; where, when When the value is 7, The function has a maximum value of 0.999, which approaches 1.
[0036] It should be noted that, This refers to the amplitude adjustment coefficient of the ln() function, which is set manually based on experience. It is used to adjust the position number WH of each recorded text in the current sentence to the basic sentiment factor. The degree of influence; when other conditions remain unchanged. The larger the basic emotional factor The greater the impact, Smaller basic emotional factors The smaller the impact.
[0037] In one possible implementation of this application embodiment, the above-mentioned S3 can be implemented by the following S301, which will be described in detail below: S301. By modifying the basic sentiment factors through the text data of each sentence, the characteristic sentiment factors of each sentence are determined, including: Extracting the basic sentiment factors of the current statement The corresponding emotion type is identified by using an emotion text database to identify text belonging to that emotion type in the current sentence, and the proportion of that text to the total text is obtained (BL). Extract the intervals between each recorded text in the current sentence, and mark the intervals from beginning to end as follows: The rate of change of the interval between two adjacent intervals is determined sequentially based on the calculation formula (3). For several of the aforementioned interval change rates The average rate of change, PGB, is obtained by calculating the average value. Based on proportional BL, average rate of change PGB, and basic sentiment factor The characteristic emotional factors of the current statement are determined by calculation formula (4). ; The calculation formula (3) is: ; The calculation formula (4) is: .
[0038] It is worth noting that this step uses the transcript of the recorded statement and the intervals between the transcripts as the basis for analysis, and comprehensively calculates the variation characteristics of basic emotional factors and speech rhythm to obtain more accurate characteristic emotional factors.
[0039] First, extract the basic sentiment factors of the current statement. The system identifies the corresponding emotion type and uses an emotion-based text database to recognize words belonging to that emotion type within the sentence. It then calculates the proportion (BL) of these emotion words in the entire sentence, reflecting the semantic intensity of the emotion. Next, the intervals between every two adjacent recorded texts in the sentence are extracted and marked. And the interval change rate is obtained using formula (3). Then for all The average value is taken to obtain PGB, which quantifies the fluctuations in speech rhythm and reveals the trends in speech rate and pauses. Finally, the proportion BL, average rate of change PGB, and basic sentiment factor are combined. The characteristic emotion factor was calculated using formula (4). This allows for the organic integration of semantic emotional intensity with changes in speech rhythm.
[0040] In practical applications, when emotions intensify, it often manifests as a faster speaking speed, shorter word intervals, and a decreased rate of change in intervals, which makes... A significant increase in speech rate reflects elevated mood; conversely, as mood decreases, speech rate slows, word intervals lengthen, and the rate of change increases, ultimately... The emotion will decrease, manifesting as a tendency towards emotional stability. Since emotions in real human communication are often not constant but fluctuate with context and inner state, this method can capture the dynamic characteristics of emotion changes with rhythm in sentences in real time and quantify them through mathematical models. This allows emotion analysis to no longer rely solely on semantics but also consider the changing patterns of speech rhythm. This analysis method, which integrates semantic proportions and changes in speech intervals, can significantly improve the accuracy and sensitivity of emotion recognition, reduce omissions and misjudgments from single-dimensional analysis, and is more sensitive to capturing changes in emotional state, especially showing strong adaptability in long sentences, variable speech rates, or emotionally unstable contexts. This invention can more comprehensively reflect the speaker's true emotional dynamics, providing more reliable data support for applications such as emotion computing, mental health monitoring, intelligent customer service emotion response, and voice interaction systems. This enhances the naturalness of human-computer interaction and the ability to understand emotions, enabling machines to make more reasonable responses and judgments when faced with different emotional states.
[0041] It is worth noting that when the proportion of words related to a certain emotion category in the text reaches or exceeds 35%, the emotion of the entire sentence can be classified into that emotion category. This step is designed as follows: A function is used to represent this situation. This function reflects the impact of changes in the proportion of emotion words on the overall emotion judgment through non-linear growth. When the proportion of emotion words (BL) is in the range of [0, 0.35], the function value increases exponentially with the increase of BL. This design fully considers the sensitivity of the overall emotion judgment when the proportion of emotion words is in a lower range. When BL exceeds 0.35, the function value is limited to 1. This design not only ensures the stability of the classification results but also avoids the over-amplification of emotion judgment. Through this ingenious design, this step achieves a good balance between the accuracy and robustness of emotion judgment.
[0042] It should be noted that when hour, The value is 0.
[0043] It should be noted that the emotional text database is based on manual settings, for example: Neutral emotions: Okay, I understand, no problem, thank you, I get it; Happiness: This is awesome! This is great! I'm so happy! I love you! Haha! Hehe! Yay! Angry: Get lost, shut up, so annoying, I'm so angry, go to hell, never stop, are you sick? Sadness: I'm so sad, heartbroken, so devastated, I want to cry, I feel like I can't take it anymore; Fear: Please stop talking, save me.
[0044] In one possible implementation of this application embodiment, the above-mentioned S4 can be implemented by the following S401 and S402, which are described in detail below: S401. Extract the emotion type from this recording, including: Extracting characteristic sentiment factors from each statement and characteristic emotional factors The corresponding emotion type is used to label each statement according to its emotion type. .
[0045] S402. Store the recording based on characteristic emotion factors and emotion type, including: Recordings of the same subject were categorized by emotion type. Categorized storage based on emotion type Set the text of each sentence to the corresponding type color, and according to the mood type. Corresponding characteristic emotion factors Determine the RGB value of the color of the current statement type; The recordings are stored as a whole, based on emotion type. Set the time bar and text for each statement in the recording to the corresponding type color, and adjust them according to the emotion type. Corresponding characteristic emotion factors Determine the RGB value of the current statement's type color; where, in the type color, when the emotion type... When the emotion is neutral, the color is off-white; when the emotion type... When feeling happy, the color is yellow; when the emotion type is yellow... When angry, the type color is red; when the emotion type is red... When feeling sad, the color is blue; when the emotion type is... When experiencing fear, the type color is black.
[0046] It should be noted that, according to emotion type Corresponding characteristic emotion factors To determine the RGB value of the color of the current statement type, the specific steps are as follows: Extract sentences sequentially to obtain the characteristic sentiment factors of the current sentence. The ratio to the standard sentiment factor is used to select the corresponding RGB value from the corresponding RGB ratio library, and the color corresponding to the RGB value is used as the time bar and text type color of the current sentence in the recording; wherein, the standard sentiment factor is obtained based on experience and can be several characteristic sentiment factors stored in the system. The average value; an RGB ratio library is established for each emotion type, for example, when the emotion type When the color is red, the corresponding RGB ratio library ranges from light red to dark red. The RGB values for each ratio are manually set.
[0047] The foregoing mainly describes the solutions of the embodiments of this application from the perspective of device implementation. It is understood that each device, such as a recording data storage device, includes at least one of the hardware structures and software modules corresponding to the execution of each function in order to achieve the above-mentioned functions. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0048] This application embodiment can divide a recording data storage device into functional units based on the above method example. For example, each function can be divided into separate functional units, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.
[0049] When using integrated units, Figure 3 A possible structural schematic diagram of a recording data storage device (referred to as communication device 30) involved in the above embodiments is shown. The communication device 30 includes a processing unit 301 and a communication unit 302, and may also include a storage medium (referred to as storage unit 303). Figure 3 The structural diagram shown can be used to illustrate the structure of a recording data storage device involved in the above embodiments.
[0050] when Figure 3 The schematic diagram shown illustrates the structure of a recording data storage device involved in the above embodiments. The processing unit 301 is used to control and manage the operation of the recording data storage device, the communication unit 302 is used for the recording data storage device to communicate with other devices, and the storage unit 303 is used to store the program code and data of the recording data storage device.
[0051] For example, communication unit 302 is used to perform noise reduction processing on the recording and obtain the text data and speech data of the noise-reduced recording; wherein, the text data includes the recorded text and the interval between the recorded text; the speech data includes the pitch, speech rate and volume of the recorded text; Processing unit 301: is used to determine the reference emotion factor of each recorded text based on the speech data, determine the basic emotion factor of each sentence based on the reference emotion factor, modify the basic emotion factor of each sentence through the text data of each sentence to determine the characteristic emotion factor of each sentence, extract the emotion type of the current recording, and store the recording based on the characteristic emotion factor and the emotion type.
[0052] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, the disclosure, and the appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple components. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.
[0053] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely illustrative descriptions of the application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.
[0054] Some of the data in the above calculation formula are obtained by removing dimensions and taking their numerical values. The calculation formula is a calculation formula that is closest to the real situation, obtained by software simulation of a large amount of collected data. The preset parameters and preset thresholds in the calculation formula are set by those skilled in the art according to the actual situation or obtained through simulation of a large amount of data.
Claims
1. A method for storing audio recording data, characterized in that, include: The recording is noise-reduced to obtain the text and speech data of the noise-reduced recording; the text data includes the intervals between the recorded text; the speech data includes the pitch, speech rate and volume of the recorded text. Reference sentiment factors for each recorded text are determined based on the speech data, and basic sentiment factors for each sentence are determined based on the reference sentiment factors. The basic sentiment factors of each sentence are corrected by analyzing the text data of each sentence to determine the characteristic sentiment factors of each sentence. Extract the emotion type from this recording and store the recording based on the characteristic emotion factors and emotion type.
2. The method for saving audio recording data according to claim 1, characterized in that, The noise reduction process for the recording includes: The recorded signal is subjected to spectral analysis to extract noise frequency band features. The noise type is then classified using support vector machine and deep neural network, and noise type labels are output. The frequency band features include high-frequency noise and low-frequency noise. The noise type labels include environmental noise, mechanical noise, and mixed human voice noise. The filter parameters are adjusted based on the noise type label and frequency band characteristics; the adjustment of the filter parameters is based on adaptive filtering algorithms, including LMS algorithm and RLS algorithm. The recording signal is divided into high-frequency, mid-frequency, and low-frequency bands, and the recording signal of each band is denoised using adaptive filter parameters that match the noise type of the corresponding frequency band.
3. The method for saving audio recording data according to claim 1, characterized in that, The acquisition of the text and voice data of the noise-reduced recording includes: Recordings are identified by timbre, and sentences with the same timbre are marked as the same recognition object; text recognition technology is used to obtain the recorded text of each recognition object and the spacing between each character in the recorded text; among them, speech recognition technology includes Speech-to-Text API and Azure Speech Services. The tone, speed, and volume of each character in the recorded text of each identified object are obtained through speech analysis tools, including Praat and Audacity.
4. The method for saving audio recording data according to claim 1, characterized in that, The determination of reference sentiment factors for each recorded text based on speech data includes: Extract the pitch, speech rate, volume, and reference sentiment factors of each recorded text from the reference recordings in the reference database; wherein, the reference database includes the pitch, speech rate, and volume of several recorded texts, as well as the reference sentiment factors set by experts based on the pitch, speech rate, and volume of the recorded texts. The pitch, speech rate, volume, and reference emotion factors of each recorded text are integrated into several sets of training data and test data. The training data is used to train the artificial intelligence model, and the test data is used to test the trained artificial intelligence model. The artificial intelligence model is adjusted according to the test results. Finally, an emotion recognition model is obtained with the pitch, speech rate, and volume of the recorded text as input and the reference emotion factors of each recorded text as output. The artificial intelligence model includes a BP neural network model and an RBF neural network model.
5. The method for saving audio recording data according to claim 1, characterized in that, The determination of the basic sentiment factors for each statement based on reference sentiment factors includes: Sequentially extract sentences from the recording, extract the position number (WH) of each recorded text within the current sentence; extract the reference sentiment factor for each text in the current sentence. The text is marked as the target text in order from back to front according to the position number WH; where i represents the emotion type number, and the value of i is in the range of [1,n], and n is the maximum value of the emotion type number; j represents the magnitude of the emotion value; the emotion types include neutral emotion, happiness, anger, sadness and fear; When the target text has a different emotion type from both of its two adjacent texts, determine whether the emotion types of the two adjacent texts are the same. If yes, change the emotion type of the target text to the emotion type of the two adjacent texts and set the base emotion factor of the target text to 0. If no, do nothing. Determine whether only one emotion type exists in the extracted statement; if yes, determine the basic emotion factor of the current statement based on formula (1). No, retrieve the number of characters corresponding to each emotion type in the current sentence. And based on the calculation formula (2), the preliminary emotion factors corresponding to various emotion types in the current sentence are determined. The largest initial emotional factor Multiply Obtain the basic sentiment factor of the current statement ;in, It is the error adjustment coefficient, and The value range of is [1,2]; The calculation formula (1) is: ； Where ZS is the total number of characters in the current statement; This is the amplitude adjustment coefficient, and The value range is [1, 1.71828]; The calculation formula (2) is: 。 6. The method for storing audio recording data according to claim 1, characterized in that, The process of determining the characteristic emotional factors of each sentence by correcting the basic emotional factors through the text data of each sentence includes: Extracting the basic sentiment factors of the current statement The corresponding emotion type is identified by using an emotion text database to identify text belonging to that emotion type in the current sentence, and the proportion of that text to the total text is obtained (BL). Extract the intervals between each recorded text in the current sentence, and mark the intervals from beginning to end as follows: The rate of change of the interval between two adjacent intervals is determined sequentially based on the calculation formula (3). For several of the aforementioned interval change rates The average rate of change, PGB, is obtained by averaging. Based on proportional BL, average rate of change PGB, and basic sentiment factor The characteristic emotional factors of the current statement are determined by calculation formula (4). ; The calculation formula (3) is: ； The calculation formula (4) is: 。 7. The method for saving audio recording data according to claim 1, characterized in that, The extraction of the emotion type from this recording includes: Extracting characteristic sentiment factors from each statement and characteristic emotional factors The corresponding emotion type is used to label each statement according to its emotion type. .
8. The method for storing audio recording data according to claim 1, characterized in that, The recording storage based on characteristic emotion factors and emotion type includes: Recordings of the same subject were categorized by emotion type. Categorized storage based on emotion type Set the text of each sentence to the corresponding type color, and according to the mood type. Corresponding characteristic emotion factors Determine the RGB value of the color of the current statement type; The recordings are stored as a whole, based on emotion type. Set the time bar and text for each statement in the recording to the corresponding type color, and adjust them according to the emotion type. Corresponding characteristic emotion factors Determine the RGB value of the current statement's type color; where, in the type color, when the emotion type... When the emotion is neutral, the color is off-white; when the emotion type... When feeling happy, the color is yellow; when the emotion type is yellow... When angry, the type color is red; when the emotion type is red... When feeling sad, the color is blue; when the emotion type is... When experiencing fear, the type color is black.
9. A recording data storage device, used to operate the recording data storage method according to any one of claims 1 to 8, characterized in that, The device includes: a communication unit and a processing unit; The communication unit is used to perform noise reduction processing on the recording and obtain the text data and voice data of the noise-reduced recording; wherein, the text data includes the recording text and the interval between the recording text; the voice data includes the pitch, speech rate and volume of the recording text; The processing unit is used to: determine reference emotion factors for each recorded text based on speech data; determine basic emotion factors for each sentence based on reference emotion factors; determine characteristic emotion factors for each sentence by correcting the basic emotion factors using text data of each sentence; extract the emotion type of the current recording; and store the recording based on the characteristic emotion factors and emotion type.
10. A storage medium, characterized in that, Used for storing computer programs, which, when executed, implement a method for saving audio recording data as described in any one of claims 1 to 8.