A method and device for generating character portrait based on intelligent speech analysis

Abstract

The present invention discloses a method and device for generating a character character portrait based on intelligent speech analysis. The method comprises: extracting feature information of audio data of known persons, generating a speech feature data pool according to the feature information; calibrating the character of each feature information in the speech feature data pool according to the character characteristics of the known persons; extracting the feature information to be identified of each user audio in the audio data pool to be analyzed; matching the feature information to be identified of each user audio with the feature information in the speech feature data pool, and determining the character category of the feature information to be identified of each user audio according to the matching result; generating a character character portrait of each user audio in the audio data pool to be analyzed according to the character category of each user audio.

Description

Technical Field

[0001] The present invention relates to the field of speech analysis technology, and more specifically, to a method and device for generating a character profile based on intelligent speech analysis. Background Art

[0002] Nowadays, customer centers contain a large amount of customer information and customer data. How to accurately find the corresponding customers from massive data has become a thorny problem. Therefore, massive user information can be processed to generate vivid user portraits. However, how to transform a rigid customer data into a more vivid customer character portrait so that the corresponding business personnel can easily grasp the customer's personality and preferences when communicating with the customer has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] In view of the deficiencies in the prior art, the present invention provides a method and device for generating a character profile based on intelligent speech analysis.

[0004] According to one aspect of the present invention, a method for generating a character profile based on intelligent speech analysis is provided, comprising:

[0005] Extract feature information of audio data of known persons, and generate a voice feature data pool based on the feature information;

[0006] According to the known personality characteristics of the person, each feature information in the voice feature data pool is calibrated;

[0007] Extracting feature information to be identified from each user's audio in the audio data pool to be analyzed;

[0008] Match the feature information to be identified of each user's audio with the feature information in the voice feature data pool, and determine the personality category of the feature information to be identified of each user's audio according to the matching result;

[0009] According to the personality category of each user's audio, a personality portrait of each user's audio in the audio data pool to be analyzed is generated.

[0010] Optionally, feature information of audio data of known persons is extracted, including:

[0011] Converting audio data of a known person into an audio signal;

[0012] Use blind source separation to separate human voice from background sound in audio signal and obtain human voice in audio data;

[0013] Extract characteristic information from human voice, where the characteristic information includes: pitch characteristics, timbre characteristics, speaking speed characteristics and tone characteristics.

[0014] Optionally, feature information in human voice is extracted, including:

[0015] Extract timbre features from human voice using Mel spectrum method;

[0016] Determine the tonal characteristics based on the high and low frequencies in the human voice;

[0017] Determine the speech rate characteristics based on the frequency speed in the human voice;

[0018] Extract emotional words from human voice and determine the tone characteristics.

[0019] Optionally, the timbre is extracted from the human voice using a Mel spectrum method, including:

[0020] Calculate the Mel spectrum of human voice;

[0021] Extract timbre features based on Mel spectrum;

[0022] The HTM model is used to calculate the HTM features of the Mel spectrum to obtain the harmonic structure of the timbre characteristics and the time domain modulation information.

[0023] Optionally, the HTM model is used to calculate the Mel spectrum HTM features to obtain the harmonic structure and time domain modulation information of the timbre features, including:

[0024] Use the HTM model to obtain the activation state of each input feature of the Mel spectrum;

[0025] Calculate the activity score of each input feature based on its activation state;

[0026] Arrange the activity scores of all input features in descending order, and select HTM features based on the preset threshold;

[0027] Calculate the total input signal value of each cell in the HTM model based on the activity score of the input features connected to each cell within the preset time window;

[0028] Activate the cell whose total input signal value is greater than a preset threshold;

[0029] Set the cell with the highest activity score within the preset time window to active state;

[0030] According to the activated and active cells, the HTM features in each preset time window of the Mel spectrum are determined to obtain the harmonic structure and time domain modulation information of the timbre characteristics.

[0031] Optionally, the calculation formula of the activation state is:

[0032] P(t)=sigmoid(sum(w*x))

[0033] Among them, P(t) represents the probability of activating the input feature, that is, the activation state, w represents the connection weight, x represents the input signal value of the input feature, and the sigmoid function maps the input signal value to between 0 and 1;

[0034] The activity score is calculated as follows:

[0035] A(t)=P(t)-k*avg(P)

[0036] Where A(t) represents the activity score of the input feature, k represents a constant, and avg(P) represents the mean activation probability of all input features;

[0037] The total input signal value of the cell is calculated as:

[0038] input(t)=sum(A(t)*w)

[0039] Where input(t) represents the input signal value of the cell, A(t) represents the activity score of the connected input feature, and w represents the connection weight.

[0040] Optionally, matching the feature information to be identified of each user audio with the feature information in the voice feature data pool, and determining the personality category of the feature information to be identified of each user audio according to the matching result, including:

[0041] Calculate the similarity between the feature information to be identified of each user's audio and all the feature information in the voice feature data pool;

[0042] Select the feature information with the highest similarity as the matching result of the feature information to be identified;

[0043] The personality category of the feature information to be identified of each user audio is determined according to the personality data calibrated by the feature information corresponding to the matching result.

[0044] According to another aspect of the present invention, there is provided a device for generating a character profile based on intelligent speech analysis, comprising:

[0045] A first extraction module is used to extract feature information of audio data of a known person and generate a voice feature data pool according to the feature information;

[0046] The calibration module is used to calibrate the personality of each feature information in the voice feature data pool according to the known personality characteristics of the person;

[0047] The second extraction module is used to extract the feature information to be identified of each user audio in the audio data pool to be analyzed;

[0048] A matching module, used to match the feature information to be identified of each user's audio with the feature information in the voice feature data pool, and determine the personality category of the feature information to be identified of each user's audio according to the matching result;

[0049] The generation module is used to generate a character profile of each user audio in the audio data pool to be analyzed according to the character category of each user audio.

[0050] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the method described in any one of the above aspects of the present invention.

[0051] According to another aspect of the present invention, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described in any one of the above aspects of the present invention.

[0052] Therefore, this application generates a voice feature data pool through intelligent voice analysis, and generates character portraits by matching the corresponding personality categories through voice analysis of the customers. It easily completes character portraits of tens of thousands of customers, and sales and business personnel can easily grasp the other party's personality during conversations with customers. Technical effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] A more complete understanding of exemplary embodiments of the present invention may be obtained by referring to the following drawings:

[0054] Figure 1 It is a flowchart of a method for generating a character profile based on intelligent speech analysis provided by an exemplary embodiment of the present invention;

[0055] Figure 2 It is a structural schematic diagram of a device for generating a character profile based on intelligent speech analysis provided by an exemplary embodiment of the present invention;

[0056] Figure 3 This is a structure of an electronic device provided by an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0057] Below, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described here.

[0058] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

[0059] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present invention are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0060] It should also be understood that, in the embodiments of the present invention, “plurality” may refer to two or more than two, and “at least one” may refer to one, two or more than two.

[0061] It should also be understood that any component, data or structure mentioned in the embodiments of the present invention can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0062] In addition, the term "and / or" in the present invention is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects before and after are in an "or" relationship.

[0063] It should also be understood that the description of the various embodiments of the present invention focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0064] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0065] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0066] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0067] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0068] Embodiments of the present invention can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0069] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0070] Exemplary Methods

[0071] Figure 1 FIG. 1 is a flow chart of a method for generating a character profile based on intelligent speech analysis provided by an exemplary embodiment of the present invention. This embodiment can be applied to electronic devices, such as Figure 1 As shown, the method 100 for generating a character profile based on intelligent speech analysis includes the following steps:

[0072] Step 101, extracting feature information of audio data of a known person, and generating a voice feature data pool according to the feature information;

[0073] Step 102, characterization is performed on each feature information in the voice feature data pool according to the known personality characteristics of the person;

[0074] Step 103, extracting feature information to be identified of each user audio in the audio data pool to be analyzed;

[0075] Step 104, matching the feature information to be identified of each user's audio with the feature information in the voice feature data pool, and determining the personality category of the feature information to be identified of each user's audio according to the matching result;

[0076] Step 105, generating a character profile of each user audio in the audio data pool to be analyzed according to the character category of each user audio.

[0077] Specifically, the customer data of the customer center includes a large number of customer image data. By extracting the customer's voice, the speaking speed, pitch, timbre, use of modal particles, etc. of the customer's voice are gradually extracted. By manually marking the personalities of familiar customers and matching them with the voice characteristics, a big data pool is formed. For example, it is analyzed that timbre A corresponds to a customer with a calm personality, timbre B corresponds to a customer with a flamboyant personality, etc., and then big data analysis is performed on the voices of unfamiliar customers to obtain a personality portrait of the unfamiliar customer.

[0078] The implementation steps are as follows:

[0079] S100, extracting the speech speed, timbre, modal particles, etc. of familiar persons through codes, extracting a large amount of voice feature information of familiar persons, and generating a voice feature data pool.

[0080] Among them, the tone can be divided into different levels, from low to high, from 1-10.

[0081] Tone: It can be distinguished based on frequency, amplitude, and waveform.

[0082] Speaking speed: It can be divided into different levels, from fast to slow, from 1-10.

[0083] Tone: mainly refers to the emotional color it reveals, namely joy, anger, sorrow, happiness, desire, hate, and fear.

[0084] Further, 1. Read the audio file and convert it into an audio signal

[0085] audio_path="audio.wav"

[0086] audio,sr=librosa.load(audio_path,sr=None)

[0087] 2. Use blind source separation algorithm to separate audio signals into human voice and background music

[0088] This removes noise from the audio data.

[0089] Further, the timbre extraction method:

[0090] The Mel spectrum method is used to convert the STFT spectrum into a Mel frequency scale, simulating the human ear's ability to perceive different frequencies, which is more in line with human voice perception. 1. Calculate the Mel spectrum:

[0091] mel_spec=librosa.feature.melspectrogram(y=audio,sr=sr).

[0092] 2. Extract timbre features:

[0093] mfcc=librosa.feature.mfcc(S=librosa.power_to_db(mel_spec)).

[0094] Among them, feature extraction is performed on audio data:

[0095] 1. Sine wave FFT: Perform short-time Fourier transform (STFT) on the audio signal s(t), and the analysis window is often the Hanning window. Get the spectrum of each frame.

[0096] 2. Data pre-emphasis: to balance the spectrum and increase the energy of high-frequency components.

[0097] 3. Equidistant logarithmic segmentation: Set the segmentation point, and divide the audio frequency f(i) into M filters MFCC_fi, i = 0, 1, 2, ... M. Each filter MFCC_fi is processed according to the trigonometric function.

[0098] 4. Triangular filter bank: Calculate the energy E_MFCC(i) of each filter in Mel scale.

[0099] 5. Take the logarithm of the signal after filter bank processing for each frame and perform discrete cosine transform to obtain MFCC.

[0100] For setting of the filter bank, the frequency f is converted into the Mel scale using the formula: mel(f)=2595log10(1+f / 700).

[0101] Set the filter bank as described above, analyze the frame and extract the MFCC, and then use these MFCCs as image features to identify and classify the audio.

[0102] 3. Calculate HTM features (HTM is a method to describe timbre by analyzing the harmonics, frequency profile and modulation characteristics of the Mel spectrum. By calculating the spectral peaks, brightness and energy characteristics of the short-time Mel spectrum, the harmonic structure and time domain modulation information of the timbre can be obtained):

[0103] magnitude = np.abs(mel_spec)

[0104] phase = np.angle(mel_spec)

[0105] chroma=librosa.feature.chroma_stft(S=magnitude,sr=sr)

[0106] chroma_cens=librosa.feature.chroma_cens(C=chroma).

[0107] Among them, the HTM feature algorithm is calculated: extracted from the original input data in the Spatial Pooling stage, and then further processed in the Temporal Pooling stage.

[0108] (1) Spatial Pooling: In the Spatial Pooling stage, the HTM model extracts features by calculating the activity (activity and competition process) of each input feature. The activity score of each input feature is calculated, and a portion of features with the highest activity score is selected as HTM features.

[0109] [1] Activate input features: First, for each input feature, calculate the sum of the connection weights of the feature and the previous time step, that is, the input signal value of the input feature. Then, by applying a nonlinear threshold function (such as the Sigmoid function or the ReLU function), the input signal value is mapped to a range between 0 and 1 to obtain the activation state of the input feature. For example, the probability of activating an input feature can be calculated using the following formula:

[0110] P(t)=sigmoid(sum(w*x))

[0111] Among them, P(t) represents the probability of activating the input feature, w represents the connection weight, and x represents the input signal value of the input feature. The sigmoid function maps the input signal value to between 0 and 1.

[0112] [2] Competition process: In order to extract the most representative features, competition between features is required. The activity score of each input feature can be calculated using the following formula:

[0113] A(t)=P(t)-k*avg(P)

[0114] Where A(t) represents the activity score of the input feature, P(t) represents the probability of activating the input feature, k represents a constant (usually 0.1 to 0.5), and avg(P) represents the mean activation probability of all input features. By subtracting the mean, the influence of the overall activation level can be eliminated and a relatively high activity can be retained.

[0115] [3] Select HTM features: Based on the activity scores, select a portion of the input features with the highest activity scores as HTM features. Features can be selected based on a pre-set threshold or ratio. For example, the 10% of input features with the highest activity scores can be selected as HTM features.

[0116] (2) Temporal Pooling: In the Temporal Pooling stage, the HTM model extracts time-related features by activating and competing cells in time. This process can be achieved by calculating the activity and connection weights between cells.

[0117] [1] Activate cells: For each cell, calculate the total input signal value of the cell based on the activity scores of the input features it is connected to. The input signal value of a cell can be calculated using the following formula:

[0118] input(t)=sum(A(t)*w)

[0119] Where input(t) represents the input signal value of the cell, A(t) represents the activity score of the connected input feature, and w represents the connection weight. The activity scores of all connected input features are multiplied by the weight and summed to obtain the input signal value of the cell.

[0120] [2] Cell activation: By comparing the input signal value of a cell with a predefined threshold, it is possible to determine whether the cell is activated. If the input signal value of the cell is greater than the threshold, it is activated. Otherwise, the cell is in an inactive state.

[0121] [3] Cell competition: When processing time series data, the HTM model needs to select the most representative active cells in the time window, and other cells are in a competitive state. A common competition mechanism is Winner-Takes-All (WTA), which selects the cell with the highest activity score and sets it to an active state, and sets other cells to an inactive state.

[0122] [4] Time Window: Temporal Pooling usually uses a fixed-size time window for feature extraction. At each time step, only cells within the time window can participate in the calculation and competition of activity scores. As time goes by, the time window will slide, allowing the model to handle constantly changing time series data.

[0123] Among them, the loss function of the HTM model usually uses a cross-entropy loss function (Cross-Entropy Loss). For example, if the true label is Y (a One-Hot encoded vector) and the predicted label is P (a probability vector), then the cross-entropy loss function can be defined as follows:

[0124] L=-1 / n*∑[y_i*log(p_i)]

[0125] Where n is the number of categories, i represents the i-th category, y_i is the i-th element in the true label vector Y, and p_i is the i-th element in the predicted vector P.

[0126] 4. Calculate the spectral peaks, brightness and modulation features, and further analyze, visualize and apply the HTM features:

[0127] spectral_peaks=librosa.feature.spectral_peak_hz(S=magnitude,sr=sr)

[0128] brightness=librosa.feature.spectral_brightness(S=magnitude)

[0129] modulation=librosa.feature.spectral_contrast(S=magnitude).

[0130] (1) Calculate the spectral peak: The spectral peak represents the highest peak value of a feature within a certain frequency range. It can be obtained by calculating the peak value of the HTM feature in the spectrum or time frequency.

[0131] [1] Perform Fourier transform on HTM features to convert them into frequency domain signals

[0132] [2] Calculate the power spectral density of the frequency domain signal

[0133] [3] Find the frequency with the maximum power on the spectrum, which is the spectrum peak

[0134] (2) Calculate brightness: Brightness represents the overall activity level of the HTM feature. It can be obtained by calculating the average or sum of the activity scores of the HTM feature over time - summing or averaging the activity scores of each time step of the HTM feature.

[0135] (3) Calculate modulation: The modulation feature represents the change of the HTM feature at different time steps. It can be obtained by calculating the variance or standard deviation of the activity score of the HTM feature - calculating the variance or standard deviation of the activity score of each time step of the HTM feature.

[0136] S200. For the person whose voice characteristics have been extracted, corresponding personality characteristics are entered according to personal impressions.

[0137] S300, matching the voice characteristics with the personalities entered by familiar persons through codes to generate a data pool.

[0138] S400. Extract the voice characteristics of unfamiliar customers, match them with the voice-personality characteristics in the data pool, and generate a personality profile of the customer.

[0139] Therefore, this application generates a voice feature data pool through intelligent voice analysis, and generates character portraits by matching the corresponding personality categories through voice analysis of the customers. It easily completes character portraits of tens of thousands of customers, and sales and business personnel can easily grasp the other party's personality during conversations with customers. Technical effect.

[0140] Exemplary Devices

[0141] Figure 2 FIG. 1 is a schematic diagram of a device for generating a character profile based on intelligent speech analysis according to an exemplary embodiment of the present invention. Figure 2 As shown, the device 200 includes:

[0142] A first extraction module 210 is used to extract feature information of audio data of a known person and generate a voice feature data pool according to the feature information;

[0143] The calibration module 220 is used to calibrate the personality of each feature information in the voice feature data pool according to the personality characteristics of the known person;

[0144] The second extraction module 230 is used to extract the feature information to be identified of each user audio in the audio data pool to be analyzed;

[0145] A matching module 240, for matching the feature information to be identified of each user's audio with the feature information in the voice feature data pool, and determining the personality category of the feature information to be identified of each user's audio according to the matching result;

[0146] The generation module 250 is used to generate a character personality portrait of each user audio in the audio data pool to be analyzed according to the character category of each user audio.

[0147] Optionally, the first extraction module 210 extracts feature information of audio data of a known person, including:

[0148] A conversion submodule, for converting audio data of a known person into an audio signal;

[0149] The separation submodule is used to separate the human voice and the background sound of the audio signal by using the blind source separation method to obtain the human voice in the audio data;

[0150] The extraction submodule is used to extract characteristic information from human voice, wherein the characteristic information includes: pitch characteristics, timbre characteristics, speaking speed characteristics and tone characteristics.

[0151] Optionally, extract submodules, including:

[0152] An extraction unit, for extracting timbre features from human voice using a mel spectrum method;

[0153] A first determining unit, used to determine the pitch characteristics according to the frequency of the human voice;

[0154] A second determining unit is used to determine a speech rate feature according to a frequency speed in a human voice;

[0155] The third determination unit is used to extract emotional words from human voice and determine the tone characteristics.

[0156] Optionally, the extraction unit comprises:

[0157] A first calculation subunit, used for calculating the Mel spectrum of human voice;

[0158] An extraction subunit, used for extracting timbre features according to the Mel spectrum;

[0159] The second calculation subunit is used to calculate the HTM features of the Mel spectrum using the HTM model to obtain the harmonic structure and time domain modulation information of the timbre features.

[0160] Optionally, the second computing subunit includes:

[0161] Use the HTM model to obtain the activation state of each input feature of the Mel spectrum;

[0162] Calculate the activity score of each input feature based on its activation state;

[0163] Arrange the activity scores of all input features in descending order, and select HTM features based on the preset threshold;

[0164] Calculate the total input signal value of each cell in the HTM model based on the activity score of the input features connected to each cell within the preset time window;

[0165] Activate the cell whose total input signal value is greater than a preset threshold;

[0166] Set the cell with the highest activity score within the preset time window to active state;

[0167] According to the activated and active cells, the HTM features in each preset time window of the Mel spectrum are determined to obtain the harmonic structure and time domain modulation information of the timbre characteristics.

[0168] Optionally, the calculation formula of the activation state is:

[0169] P(t)=sigmoid(sum(w*x))

[0170] Among them, P(t) represents the probability of activating the input feature, that is, the activation state, w represents the connection weight, x represents the input signal value of the input feature, and the sigmoid function maps the input signal value to between 0 and 1;

[0171] The activity score is calculated as follows:

[0172] A(t)=P(t)-k*avg(P)

[0173] Where A(t) represents the activity score of the input feature, k represents a constant, and avg(P) represents the mean activation probability of all input features;

[0174] The total input signal value of the cell is calculated as:

[0175] input(t)=sum(A(t)*w)

[0176] Where input(t) represents the input signal value of the cell, A(t) represents the activity score of the connected input feature, and w represents the connection weight.

[0177] Optionally, the matching module 240 includes:

[0178] A calculation submodule, used to calculate the similarity between the feature information to be identified of each user's audio and all the feature information in the voice feature data pool;

[0179] A selection submodule is used to select the feature information with the highest similarity as the matching result of the feature information to be identified;

[0180] The determination submodule is used to determine the personality category of the feature information to be identified of each user audio according to the personality data calibrated by the feature information corresponding to the matching result.

[0181] Exemplary Electronic Devices

[0182] Figure 3 This is a structure of an electronic device provided by an exemplary embodiment of the present invention. Figure 3 As shown, the electronic device 30 includes one or more processors 31 and a memory 32 .

[0183] The processor 31 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0184] The memory 32 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 31 may run the program instructions to implement the methods of the software programs of the various embodiments of the present invention described above and / or other desired functions. In one example, the electronic device may also include: an input device 33 and an output device 34, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0185] In addition, the input device 33 may also include, for example, a keyboard, a mouse, and the like.

[0186] The output device 34 can output various information to the outside. The output device 34 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto.

[0187] Of course, to simplify, Figure 3 Only some of the components related to the present invention in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device may also include any other appropriate components.

[0188] Exemplary computer program products and computer-readable storage media

[0189] In addition to the above-mentioned methods and devices, an embodiment of the present invention may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present invention described in the above-mentioned "Exemplary Method" section of this specification.

[0190] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the embodiments of the present invention, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0191] In addition, an embodiment of the present invention may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method for information mining of historical change records according to various embodiments of the present invention described in the above “Exemplary Method” section of this specification.

[0192] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, system or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0193] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details disclosed above are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0194] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0195] The block diagrams of the devices, systems, equipment, and systems involved in the present invention are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, systems, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The word "such as" used here refers to the phrase "such as but not limited to", and can be used interchangeably with it.

[0196] The method and system of the present invention may be implemented in many ways. For example, the method and system of the present invention may be implemented by software, hardware, firmware or any combination of software, hardware, firmware. The above order of steps for the method is only for illustration, and the steps of the method of the present invention are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present invention may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present invention. Thus, the present invention also covers a recording medium storing a program for executing the method according to the present invention.

[0197] It should also be noted that in the system, device and method of the present invention, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. The above description of the disclosed aspects is provided to enable any technician in the field to make or use the present invention. Various modifications to these aspects are very obvious to those skilled in the art, and the general principles defined here can be applied to other aspects without departing from the scope of the present invention. Therefore, the present invention is not intended to be limited to the aspects shown here, but in accordance with the widest range consistent with the principles and novel features disclosed here.

[0198] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present invention to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for generating a character portrait based on intelligent speech analysis, characterized in that: include: Extracting feature information of audio data of known persons, and generating a voice feature data pool according to the feature information; Performing personality calibration on each feature information in the voice feature data pool according to the personality characteristics of the known person; Extracting feature information to be identified from each user's audio in the audio data pool to be analyzed; Matching the feature information to be identified of each user's audio with the feature information in the voice feature data pool, and determining the personality category of the feature information to be identified of each user's audio according to the matching result; Generate a character profile of each user audio in the audio data pool to be analyzed according to the character category of each user audio; Extract feature information of audio data of known persons, including: converting the audio data of the known person into an audio signal; Separating the human voice from the background sound in the audio signal by using a blind source separation method to obtain the human voice in the audio data; Extracting the characteristic information from the human voice, wherein the characteristic information includes: pitch characteristics, timbre characteristics, speech speed characteristics, and tone characteristics; Extracting the characteristic information from the human voice includes: Extracting the timbre feature from the human voice using a Mel spectrum method; Determining the pitch characteristics according to the frequency height in the human voice; Determining the speech rate feature according to the frequency speed in the human voice; Extracting emotional words from the human voice to determine the tone features; Extracting the timbre from the human voice using a Mel spectrum method, comprising: Calculating the Mel spectrum of the human voice; Extracting the timbre feature according to the Mel spectrum; Calculate the HTM features of the Mel spectrum using the HTM model to obtain the harmonic structure and time domain modulation information of the timbre features; The HTM model is used to calculate the Mel spectrum HTM feature to obtain the harmonic structure and time domain modulation information of the timbre feature, including: Using the HTM model to obtain the activation state of each input feature of the Mel spectrum; Calculating an activity score for each input feature according to the activation state of each input feature; Arrange the activity scores of all input features in descending order, and select the HTM feature according to a preset threshold; Calculating the total input signal value of each cell in the HTM model according to the activity score of the input feature of each cell connection within a preset time window; Activate the cell whose total input signal value is greater than a preset threshold; Setting the cell with the highest activity score within the preset time window to an active state; Determine the HTM features in each preset time window of the Mel spectrum according to the cells in the activation state and the active state, and obtain the harmonic structure and the time domain modulation information of the timbre feature; The calculation formula of the activation state is: P(t) = sigmoid(sum(w x)) Among them, P(t) represents the probability of activating the input feature, that is, the activation state, w represents the connection weight, x represents the input signal value of the input feature, and the sigmoid function maps the input signal value to between 0 and 1; The calculation formula of the activity score is: A(t) = P(t) - k avg(P) Where A(t) represents the activity score of the input feature, k represents a constant, and avg(P) represents the mean activation probability of all input features; The total input signal value of the cell is calculated as: input(t) = sum(A(t) w) Where input(t) represents the input signal value of the cell, A(t) represents the activity score of the connected input feature, and w represents the connection weight.

2. The method according to claim 1, characterized in that Matching the feature information to be identified of each user audio with the feature information in the voice feature data pool, and determining the personality category of the feature information to be identified of each user audio according to the matching result, including: Calculate the similarity between the feature information to be identified of each user audio and all the feature information in the voice feature data pool; Selecting the feature information with the highest similarity as the matching result of the feature information to be identified; The personality category of the to-be-identified characteristic information of each user audio is determined according to the personality data calibrated by the characteristic information corresponding to the matching result.

3. A device for generating a character portrait based on intelligent speech analysis, characterized in that: include: A first extraction module, used to extract feature information of audio data of a known person, and generate a voice feature data pool according to the feature information; A calibration module, used for calibrating the personality of each feature information in the voice feature data pool according to the personality characteristics of the known person; The second extraction module is used to extract the feature information to be identified of each user audio in the audio data pool to be analyzed; A matching module, used for matching the feature information to be identified of each user's audio with the feature information in the voice characteristic data pool, and determining the personality category of the feature information to be identified of each user's audio according to the matching result; A generating module, used to generate a character character portrait of each user audio in the audio data pool to be analyzed according to the character category of each user audio; The first extraction module extracts feature information of the audio data of a known person, including: A conversion submodule, used for converting the audio data of the known person into an audio signal; A separation submodule, used to separate the human voice and background sound of the audio signal by using a blind source separation method, so as to obtain the human voice in the audio data; An extraction submodule, used to extract the feature information in the human voice, wherein the feature information includes: pitch feature, timbre feature, speech speed feature and tone feature; Extract submodules, including: An extraction unit, used for extracting the timbre feature from the human voice using a Mel spectrum method; A first determining unit, configured to determine the pitch feature according to the frequency of the human voice; A second determining unit, configured to determine the speech rate feature according to the frequency speed in the human voice; A third determining unit is used to extract the emotional words in the human voice and determine the tone feature; Extraction unit, comprising: A first calculation subunit, used for calculating the Mel spectrum of the human voice; An extraction subunit, used for extracting the timbre feature according to the Mel spectrum; The second calculation subunit is used to calculate the HTM features of the Mel spectrum using the HTM model to obtain the harmonic structure and time domain modulation information of the timbre features; The second computing subunit includes: Using the HTM model to obtain the activation state of each input feature of the Mel spectrum; Calculating an activity score for each input feature according to the activation state of each input feature; Arrange the activity scores of all input features in descending order, and select the HTM feature according to a preset threshold; Calculating the total input signal value of each cell in the HTM model according to the activity score of the input feature of each cell connection within a preset time window; Activate the cell whose total input signal value is greater than a preset threshold; Setting the cell with the highest activity score within the preset time window to an active state; Determine the HTM features in each preset time window of the Mel spectrum according to the cells in the activation state and the active state, and obtain the harmonic structure and the time domain modulation information of the timbre feature; The calculation formula of the activation state is: P(t) = sigmoid(sum(w x)) Among them, P(t) represents the probability of activating the input feature, that is, the activation state, w represents the connection weight, x represents the input signal value of the input feature, and the sigmoid function maps the input signal value to between 0 and 1; The calculation formula of the activity score is: A(t) = P(t) - k avg(P) Where A(t) represents the activity score of the input feature, k represents a constant, and avg(P) represents the mean activation probability of all input features; The total input signal value of the cell is calculated as: input(t) = sum(A(t) w) Where input(t) represents the input signal value of the cell, A(t) represents the activity score of the connected input feature, and w represents the connection weight.

4. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the method according to any one of claims 1 to 2.

5. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1-2 above.

Images