Emotion recognition method, device, equipment and storage medium based on high-frequency electrocardiogram signal

Abstract

The present application relates to an emotion recognition method, device, equipment and storage medium based on high-frequency electrocardiogram signals. The method comprises: segmenting the electrocardiogram signal of the target object to obtain an electrocardiogram signal sequence; different electrocardiogram signal wavelets include QRS wave groups and other wavelets; extracting high-frequency features of the QRS wave groups and other wavelets respectively to obtain high-frequency QRS wave group features and other high-frequency wavelet features; fusing the wavelet coding features of each electrocardiogram signal wavelet, the position coding of each electrocardiogram signal wavelet, the high-frequency QRS wave group features and other high-frequency wavelet features to obtain a fused feature sequence; determining the position weight of each electrocardiogram signal wavelet in the electrocardiogram signal sequence; extracting context features of the fused feature sequence, performing attention processing based on the position weight during the extraction process to obtain the context features of the electrocardiogram signal; and identifying the emotional state based on the context features. The use of this method can effectively improve the accuracy of emotion recognition based on electrocardiogram signals.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an emotion recognition method, device, equipment and storage medium based on high-frequency electrocardiogram signals. Background Art

[0002] Currently, ECG signals have been widely used in the health field, including fatigue detection and emotion recognition. For emotion recognition, the traditional solution is to randomly or evenly divide the ECG signals into multiple ECG signal segments, then directly input these ECG signals into the transformer model for ECG feature extraction, and then use the extracted ECG features for emotion recognition.

[0003] However, the transformer model directly extracts ECG features from ECG signal segments obtained by random or average segmentation of ECG signals for emotion recognition, which may lead to low accuracy of emotion recognition. Summary of the invention

[0004] Based on this, it is necessary to provide an emotion recognition method, device, equipment and storage medium based on high-frequency electrocardiogram signals to address the above technical problems, which can effectively improve the accuracy of emotion recognition based on electrocardiogram signals.

[0005] In a first aspect, the present application provides an emotion recognition method based on high-frequency electrocardiogram signals, the method comprising:

[0006] Segmenting the target object's ECG signal to obtain an ECG signal sequence composed of different ECG signal sub-waves; the different ECG signal sub-waves include QRS wave groups and other sub-waves;

[0007] Extracting high-frequency features from the QRS complex and the other sub-waves respectively to obtain high-frequency QRS complex features and other high-frequency sub-wave features;

[0008] The wavelet coding features of each of the ECG signal wavelets, the position coding of each of the ECG signal wavelets, the high-frequency QRS complex features and the other high-frequency wavelet features are fused to obtain a fused feature sequence;

[0009] Determining the position weight of each ECG signal wavelet in the ECG signal sequence;

[0010] Extracting context features from the fused feature sequence, and performing attention processing based on the position weights during the extraction process to obtain context features of the electrocardiogram signal;

[0011] The emotional state of the target object is identified according to the contextual features.

[0012] In one embodiment, segmenting the target object's ECG signal to obtain an ECG signal sequence composed of different ECG signal wavelets includes:

[0013] Determine the segmentation points of each sub-wave;

[0014] Dividing the electrocardiogram signal into a plurality of electrocardiogram signal sub-waves based on each of the segmentation points;

[0015] The multiple ECG signal wavelets are sorted according to the acquisition time of the ECG signal to obtain an ECG signal sequence composed of different ECG signal wavelets.

[0016] In one embodiment, determining the position weight of each ECG signal wavelet in the ECG signal sequence comprises:

[0017] Determining the relative distance between each ECG signal wavelet in the ECG signal sequence;

[0018] The position weight of the electrical signal wavelet of each heart is determined according to the relative distance.

[0019] In one embodiment, the wavelet coding features of each of the ECG signal wavelets, the position coding of each of the ECG signal wavelets, the high-frequency QRS complex features and the other high-frequency wavelet features are fused to obtain a fused feature sequence, which includes:

[0020] Performing encoding processing on each ECG signal wavelet in the ECG signal sequence to obtain a wavelet encoding feature of each ECG signal wavelet;

[0021] Performing position coding on each ECG signal wavelet in the ECG signal sequence to obtain a position coding of each ECG signal wavelet;

[0022] The wavelet coding features of each of the ECG signal wavelets, the position code, the high-frequency QRS complex features and the other high-frequency wavelet features are fused to obtain a fused feature sequence;

[0023] The extracting context features from the weighted feature sequence to obtain context features includes:

[0024] The fused feature sequence is input into a transformer model for context feature extraction, and during the extraction process, the position weight is used for attention processing through a multi-head attention mechanism in the transformer model to obtain the context feature of the electrocardiogram signal.

[0025] In one embodiment, after identifying the emotional state of the target object according to the context feature, the method further includes:

[0026] When the emotional state meets the preset state condition, generating a prompt word according to the emotional state;

[0027] The prompt words are input into a large language model to generate guidance information for emotion regulation using the large language model.

[0028] In one embodiment, the method further comprises:

[0029] Searching a plurality of music pieces that match the emotional state from a music library as music pieces to be recommended;

[0030] Obtaining basic data and historical listening data of the target object;

[0031] Performing feature extraction on the basic data and the historical listening data to obtain interest features of the target object;

[0032] Extracting music features of the music to be recommended from relevant data of the music to be recommended;

[0033] Determining a recommendation score of the music to be recommended according to the interest feature and the music feature;

[0034] Among the music to be recommended, at least one music whose recommendation score meets a preset score condition is selected as target music;

[0035] The target music is recommended to the target object.

[0036] In a second aspect, the present application also provides an emotion recognition device based on high-frequency electrocardiogram signals, the device comprising:

[0037] A segmentation module, used for segmenting the ECG signal of the target object to obtain an ECG signal sequence composed of different ECG signal sub-waves; the different ECG signal sub-waves include QRS wave groups and other sub-waves;

[0038] A first extraction module is used to extract high-frequency features of the QRS complex and other sub-waves to obtain high-frequency QRS complex features and other high-frequency sub-wave features;

[0039] A fusion module, used for fusing the wavelet coding features of each of the ECG signal wavelets, the position coding of each of the ECG signal wavelets, the high-frequency QRS complex features and the other high-frequency wavelet features to obtain a fused feature sequence;

[0040] A determination module, used to determine the position weight of each ECG signal wavelet in the ECG signal sequence;

[0041] A second extraction module is used to extract context features from the fused feature sequence, and perform attention processing based on the position weight during the extraction process to obtain context features of the electrocardiogram signal;

[0042] The recognition module is used to recognize the emotional state of the target object according to the context feature.

[0043] In one embodiment, the segmentation module is also used to determine the segmentation points of each sub-wave; divide the ECG signal into multiple ECG signal sub-waves based on each segmentation point; sort the multiple ECG signal sub-waves according to the acquisition time of the ECG signal to obtain an ECG signal sequence composed of different ECG signal sub-waves.

[0044] In one embodiment, the determination module is further used to determine the relative distance between each ECG signal wavelet in the ECG signal sequence; and determine the position weight of each ECG signal wavelet according to the relative distance.

[0045] In one embodiment, the fusion module is further used to encode each ECG signal wavelet in the ECG signal sequence to obtain the wavelet encoding feature of each ECG signal wavelet; perform position encoding on each ECG signal wavelet in the ECG signal sequence to obtain the position encoding of each ECG signal wavelet; fuse the wavelet encoding feature of each ECG signal wavelet, the position encoding, the high-frequency QRS complex feature and the other high-frequency wavelet features to obtain a fused feature sequence;

[0046] The second extraction module is also used to input the fused feature sequence into the transformer model for context feature extraction, and during the extraction process, the multi-head attention mechanism in the transformer model is used to perform attention processing using the position weight to obtain the context feature of the electrocardiogram signal.

[0047] In one embodiment, the device further comprises:

[0048] The emotion guidance module is used to generate prompt words according to the emotional state when the emotional state meets the preset state conditions; input the prompt words into the large language model to generate guidance information for emotion regulation using the large language model.

[0049] In one embodiment, the device further comprises:

[0050] The recommendation module is used to search for multiple music matching the emotional state from a music library as music to be recommended; obtain basic data and historical listening data of the target object; perform feature extraction on the basic data and the historical listening data to obtain interest features of the target object; extract music features of the music to be recommended from relevant data of the music to be recommended; determine a recommendation score of the music to be recommended based on the interest features and the music features; select at least one music whose recommendation score meets a preset score condition from the music to be recommended as target music; and recommend the target music to the target object.

[0051] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the emotion recognition method based on high-frequency electrocardiogram signals when executing the computer program.

[0052] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the emotion recognition method based on high-frequency electrocardiogram signals are implemented.

[0053] In a fifth aspect, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the emotion recognition method based on high-frequency electrocardiogram signals are implemented.

[0054] The above-mentioned emotion recognition method, device, computer equipment, storage medium and computer program product based on high-frequency ECG signals obtain an ECG signal sequence composed of QRS wave groups and other sub-waves by segmenting the ECG signal of the target object, thereby following the law of the ECG signal, and therefore can extract pure high-frequency QRS wave group features and other high-frequency sub-wave features from the QRS wave group and other sub-waves, which is beneficial to improving the accuracy of emotional state recognition; in addition, the wavelet coding features of each ECG signal sub-wave, the position coding of each ECG signal sub-wave, the high-frequency QRS wave group features and other high-frequency sub-wave features are fused to obtain a fused feature sequence, thereby enriching the features, which is also beneficial to improving the accuracy of emotional state recognition; moreover, the position weight of each ECG signal sub-wave in the ECG signal sequence is determined, context feature extraction is performed on the fused feature sequence, and attention processing is performed based on the position weight during the extraction process to obtain the context feature of the ECG signal, so that after the attention processing, the context feature can also be ensured to carry position information, so that the emotional state of the target object can be accurately identified according to the context feature, which can effectively improve the recognition accuracy of the emotional state. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a diagram of an application environment of an emotion recognition method based on high-frequency electrocardiogram signals in one embodiment;

[0056] Figure 2 is a flow chart of an emotion recognition method based on high-frequency electrocardiogram signals in one embodiment;

[0057] Figure 3 is a structural block diagram of an emotion recognition device based on high-frequency electrocardiogram signals in one embodiment;

[0058] Figure 4 is a structural block diagram of an emotion recognition device based on high-frequency electrocardiogram signals in another embodiment;

[0059] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] It should be noted that in the following description, the terms "first and second" are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first and second" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0062] The emotion recognition method based on high-frequency electrocardiogram signals provided in the embodiments of the present application can be applied to Figure 1 In the application environment shown, the terminal 101 communicates with the server 102 via a network. The data storage system can store data that the server 102 needs to process, such as electrocardiogram signals. The data storage system can be integrated on the server 102, or placed on the cloud or other network servers.

[0063] The terminal 101 or the server 102 segments the ECG signal of the target object to obtain an ECG signal sequence composed of different ECG signal sub-waves; the different ECG signal sub-waves include QRS wave groups and other sub-waves; high-frequency features are extracted from the QRS wave groups and other sub-waves respectively to obtain high-frequency QRS wave group features and other high-frequency sub-wave features; the wavelet coding features of each ECG signal sub-wave, the position coding of each ECG signal sub-wave, the high-frequency QRS wave group features and other high-frequency sub-wave features are fused to obtain a fused feature sequence; the position weight of each ECG signal sub-wave in the ECG signal sequence is determined; context feature extraction is performed on the fused feature sequence, and attention processing is performed based on the position weight during the extraction process to obtain context features of the ECG signal; the emotional state of the target object is identified according to the context features.

[0064] The terminal 101 may be an electrocardiogram detection device, or a smart phone, a tablet computer, a laptop computer, a desktop computer, an Internet of Things device, or a portable wearable device.

[0065] The server 102 may be an independent physical server or a service node in a blockchain system. A peer-to-peer network is formed between the service nodes in the blockchain system. The peer-to-peer protocol is an application layer protocol running on the Transmission Control Protocol (TCP). In addition, the server 102 may also be a server cluster composed of multiple physical servers, and may be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0066] In one embodiment, Figure 2 As shown in FIG. , a method for emotion recognition based on high-frequency electrocardiogram signals is provided. The method can be Figure 1 The method is performed by a server or a terminal in the embodiment, or by a server and a terminal in collaboration. Figure 1 The following steps are used as an example to illustrate the terminal execution in:

[0067] S201, segmenting the ECG signal of the target object to obtain an ECG signal sequence composed of different ECG signal wavelets; the different ECG signal wavelets include QRS complexes and other wavelets.

[0068] The target object may be a person or an animal. The electrocardiogram signal may be an electrocardiogram or an electrocardiogram collected in real time or at a certain time in the past.

[0069] The QRS complex consists of Q wave, R wave and S wave, representing the depolarization process of the left and right ventricles; other sub-waves can be P wave and T wave, the first half of the P wave represents the depolarization of the right atrium, and the second half is the depolarization of the left atrium; the T wave is the process of rapid repolarization of the left and right ventricles.

[0070] In one embodiment, before S201, the terminal may collect an ECG signal of the target object, or receive an ECG signal sent by an ECG detection device.

[0071] Specifically, when the terminal is an ECG detection device, the terminal can collect ECG signals when the target object is in a resting state and / or in a moving state; or, when the terminal is not an ECG detection device, the terminal can receive an ECG signal sent by the ECG detection device, which is collected by the ECG collection device when the target object is in a resting state and / or in a moving state. For example, the ECG signal can be an ECG signal collected by the ECG detection device when the user is in a resting state, and / or an ECG signal collected by the user in a moving state.

[0072] The ECG signal is collected when the target object is in motion, which can be performed when the target object is in light or intense exercise, or when the target object is in load exercise. Therefore, through the solution of the present application, the emotional state of the target object can also be detected when the target object is in motion. It should be noted that the above-mentioned ECG detection can be a dynamic ECG detection.

[0073] In one embodiment, after S201, the terminal may filter the ECG signal of the target object, such as filtering out noise signals in the ECG signal to obtain a denoised ECG signal, and apply the denoised ECG signal to a subsequent emotion recognition scheme.

[0074] In one embodiment, the terminal may determine the segmentation points of each sub-wave, divide the ECG signal into multiple ECG signal sub-waves based on the segmentation points, sort the multiple ECG signal sub-waves according to the acquisition time of the ECG signal, and obtain an ECG signal sequence composed of different ECG signal sub-waves.

[0075] The segmentation points may be the endpoints corresponding to each sub-wave in the ECG signal. The segmentation points may be identified based on waveform features or based on an ECG recognition model. The ECG recognition model may be a neural network model trained using ECG signal samples and corresponding labels.

[0076] S202, extracting high-frequency features of the QRS complex and other sub-waves respectively to obtain high-frequency QRS complex features and other high-frequency sub-wave features.

[0077] Among them, the high-frequency QRS wave group features include the length features, amplitude features, morphological features and distance features between different sub-waves of the high-frequency QRS wave group (such as the distance features between the R waves in each adjacent two QRS wave groups). Considering that the distance between R waves can reflect the corresponding emotions to a certain extent, when extracting the high-frequency QRS wave group features, incorporating the distance features between the R waves into the range of feature extraction can help improve the accuracy of emotion recognition. The high-frequency QRS wave group refers to the high-frequency component in the QRS wave group, which can refer to a QRS wave group with a frequency above 100HZ, for example, it can refer to a QRS wave group with a frequency in the range of 150HZ-250HZ.

[0078] Other high-frequency sub-wave features include length features, amplitude features and morphological features of other high-frequency sub-waves, such as length features, amplitude features and morphological features of high-frequency P waves and high-frequency T waves. Other high-frequency sub-waves refer to high-frequency components of other sub-waves, such as high-frequency components of P waves and T waves, and may refer to P waves and T waves with frequencies above 100 Hz, for example, may refer to P waves and T waves with frequencies in the range of 150 Hz-250 Hz.

[0079] In one embodiment, in order to extract high-frequency components more accurately and effectively, before performing feature extraction, the terminal may also perform gain or power amplification processing (i.e., power amplification) on the QRS complex and other sub-waves in the ECG signal sequence; or, perform power amplification processing on the ECG signal of the target object, and then segment the ECG signal of the target object to obtain an ECG signal sequence after power amplification.

[0080] S203, fusing the wavelet coding features of each ECG signal wavelet, the position coding of each ECG signal wavelet, the high-frequency QRS complex features and other high-frequency wavelet features to obtain a fused feature sequence.

[0081] The position code may be a position code obtained by performing relative position coding on each ECG signal wavelet in the ECG signal sequence.

[0082] In one embodiment, the terminal encodes each ECG signal wavelet in the ECG signal sequence to obtain the wavelet encoding feature of each ECG signal wavelet; performs position encoding on each ECG signal wavelet in the ECG signal sequence to obtain the position encoding of each ECG signal wavelet; and fuses the wavelet encoding feature, position encoding, high-frequency QRS wave group feature and other high-frequency wavelet features of each ECG signal wavelet to obtain a fused feature sequence.

[0083] The position coding may be obtained in the following manner: the terminal may use the following position coding formula to perform relative position coding on each ECG signal wavelet in the ECG signal sequence to obtain the corresponding position coding; wherein the position coding formula is as follows:

[0084]

[0085]

[0086] in, Indicates the position of the ECG signal wavelet in the ECG signal sequence. Indicates distance position Positions with an even number of distance units, Indicates distance position Positions with an odd number of distance units, Represents the dimension of the position encoding.

[0087] In one embodiment, the fusion process may include the following steps: the terminal may first fuse the wavelet coding features, high-frequency QRS complex features, and other high-frequency wavelet features of each ECG signal wavelet, and then weight the obtained fusion results using the position coding of each ECG signal wavelet to obtain a fusion feature sequence.

[0088] S204, determining the position weight of each ECG signal wavelet in the ECG signal sequence.

[0089] The position weight can represent the correlation between different ECG signal wavelets, and can reflect the influence between ECG signal wavelets at different positions.

[0090] In one embodiment, the terminal can determine the relative distance between each ECG signal sub-wave in the ECG signal sequence; determine the position weight of each ECG signal sub-wave according to the relative distance; in addition, other methods can be used to determine the position weight, such as using relative position coding to determine the position weight.

[0091] S205, extracting context features from the fused feature sequence, and performing attention processing based on position weights during the extraction process to obtain context features of the ECG signal.

[0092] Among them, when performing attention processing, a pre-trained model based on the transformer architecture (referred to as the transformer model) can be used. In actual application, it can be a BERT (Bidirectional Encoder Representations from Transformers) model.

[0093] The transformer model contains multiple layers of encoders and decoders, each of which consists of multiple attention mechanism modules and feedforward neural network modules.

[0094] In one embodiment, the terminal can input the fused feature sequence into the transformer model for context feature extraction, and during the extraction process, use the multi-head attention mechanism in the transformer model to perform attention processing using position weights to obtain the context features of the ECG signal.

[0095] For example, the fused feature sequence is input into the transformer model for context feature extraction, and in the process of feature extraction, the attention weight is calculated based on the position weight, the query matrix and the key matrix, and then the context feature of the ECG signal is generated using the attention weight and the value matrix. Alternatively, the attention weight is calculated based on the query matrix and the key matrix, and the value matrix is ​​weighted using the position weight, and then the context feature of the ECG signal is generated using the attention weight and the weighted value matrix.

[0096] For the calculation of the query matrix, key matrix and value matrix, the following method can be referred to: the terminal uses the corresponding linear transformation matrix to perform linear transformation on the fused feature sequence to obtain the query matrix, key matrix and value matrix. The expression of the linear transformation is as follows:

[0097]

[0098]

[0099]

[0100] The above represents the query matrix, represents the key matrix, represents the value matrix, Table represents the fusion feature sequence, , and are different linear transformation matrices respectively.

[0101] S206, identifying the emotional state of the target object according to the context features.

[0102] In one embodiment, after obtaining the context features of the ECG signal, the terminal may decode the context features of the ECG signal through a decoder in the transformer model to obtain the emotional state of the target object.

[0103] In the above embodiment, the ECG signal of the target object is segmented to obtain an ECG signal sequence composed of a QRS wave group and other sub-waves, thereby following the law of the ECG signal. Therefore, pure high-frequency QRS wave group features and other high-frequency sub-wave features can be extracted from the QRS wave group and other sub-waves, which is beneficial to improving the accuracy of emotional state recognition; in addition, the wavelet coding features of each ECG signal sub-wave, the position coding of each ECG signal sub-wave, the high-frequency QRS wave group features and other high-frequency sub-wave features are fused to obtain a fused feature sequence, thereby enriching the features and also helping to improve the accuracy of emotional state recognition; furthermore, the position weight of each ECG signal sub-wave in the ECG signal sequence is determined, context feature extraction is performed on the fused feature sequence, and attention processing is performed based on the position weight during the extraction process to obtain the context feature of the ECG signal, thereby ensuring that the context feature carries position information even after the attention processing. Therefore, the emotional state of the target object can be accurately identified based on the context feature, which can effectively improve the recognition accuracy of the emotional state.

[0104] In one embodiment, after identifying the emotional state of the target object, the terminal can generate guidance information according to the emotional state, so as to adjust the target object's emotions using the guidance information. Specifically, when the emotional state meets the preset state condition, the terminal generates a prompt word according to the emotional state; the prompt word is input into the large language model, so as to generate guidance information for emotion adjustment using the large language model.

[0105] The guidance information may be in the form of text, video or audio.

[0106] For example, when the terminal detects that the user is in a depressed state, it can generate a prompt word based on the user's emotional state, such as please help generate a funny story, and then input the prompt word into the large language model. The large language model generates a corresponding story based on the prompt word, and the story can be presented in text, voice or video form.

[0107] In the above embodiment, guidance information is generated according to the emotional state of the target object, so that effective emotion guidance information can be generated using a large language model. Emotional guidance of the target object can help improve or enhance the target object's emotions and enable the target object to achieve effective emotion control.

[0108] In one embodiment, after identifying the emotional state of the target object, the terminal can recommend music of interest in combination with the interests and hobbies of the target object. Specifically, the terminal can search for multiple music that matches the emotional state from the music library as music to be recommended; obtain the basic data and historical listening data of the target object; perform feature extraction on the basic data and historical listening data to obtain the interest features of the target object; extract the music features of the music to be recommended from the relevant data of the music to be recommended; determine the recommendation score of the music to be recommended based on the interest features and the music features; select at least one music whose recommendation score meets the preset score conditions from the music to be recommended as the target music; and recommend the target music to the target object.

[0109] The basic information may include the target's gender, age, and other basic information. The related data may include the singer's name, music style, and tune.

[0110] For example, when a user uses a wearable ECG detection device to detect his or her emotional state while listening to music, multiple pieces of music that match the emotional state can be found in the music library as music to be recommended, and the user's basic data and historical listening data can be obtained, and features can be extracted from the basic data and historical listening data to obtain the user's interest features; feature extraction can be performed on information such as the singer's name, music style and melody of the recommended music to obtain the music features of the music to be recommended; the score of the music to be recommended is determined based on the interest features and music features, and then music with higher scores (such as topK) is selected from the music to be recommended as target music to be recommended to the user, such as displaying the target music on the homepage of the music application. In addition, the user can be notified by voice.

[0111] In the above embodiment, music is recommended in combination with the emotional state and interest characteristics of the target object. On the one hand, music that the user is interested in can be recommended and the recommended music can be ensured to match the emotional atmosphere of the target object. On the other hand, the playing time and playing rate of the music can be increased.

[0112] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0113] Based on the same inventive concept, the embodiment of the present application also provides an emotion recognition device based on high-frequency electrocardiogram signals for implementing the above-mentioned emotion recognition method based on high-frequency electrocardiogram signals. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more embodiments of the emotion recognition device based on high-frequency electrocardiogram signals provided below can refer to the limitations of the emotion recognition method based on high-frequency electrocardiogram signals above, and will not be repeated here.

[0114] In one embodiment, Figure 3 As shown, an emotion recognition device based on high-frequency electrocardiogram signal is provided, comprising: a segmentation module 301, a first extraction module 302, a fusion module 303, a determination module 304, a second extraction module 305 and a recognition module 306, wherein:

[0115] Segmentation module 301, used to segment the ECG signal of the target object to obtain an ECG signal sequence composed of different ECG signal sub-waves; the different ECG signal sub-waves include QRS wave groups and other sub-waves;

[0116] The first extraction module 302 is used to extract high-frequency features of the QRS complex and other sub-waves respectively to obtain high-frequency QRS complex features and other high-frequency sub-wave features;

[0117] A fusion module 303 is used to fuse the wavelet coding features of each ECG signal wavelet, the position coding of each ECG signal wavelet, the high-frequency QRS complex features and other high-frequency wavelet features to obtain a fused feature sequence;

[0118] A determination module 304 is used to determine the position weight of each ECG signal wavelet in the ECG signal sequence;

[0119] The second extraction module 305 is used to extract context features from the fused feature sequence, and perform attention processing based on position weights during the extraction process to obtain context features of the ECG signal;

[0120] The identification module 306 is used to identify the emotional state of the target object according to the context features.

[0121] In one embodiment, the segmentation module 301 is also used to determine the segmentation points of each sub-wave; divide the ECG signal into multiple ECG signal sub-waves based on each segmentation point; sort the multiple ECG signal sub-waves according to the acquisition time of the ECG signal to obtain an ECG signal sequence composed of different ECG signal sub-waves.

[0122] In one embodiment, the determination module 304 is further configured to determine the relative distance between each ECG signal wavelet in the ECG signal sequence; and determine the position weight of each ECG signal wavelet according to the relative distance.

[0123] In one embodiment, the fusion module 303 is further used to encode each ECG signal wavelet in the ECG signal sequence to obtain the wavelet encoding feature of each ECG signal wavelet; perform position encoding on each ECG signal wavelet in the ECG signal sequence to obtain the position encoding of each ECG signal wavelet; fuse the wavelet encoding feature, position encoding, high-frequency QRS wave group feature and other high-frequency wavelet features of each ECG signal wavelet to obtain a fused feature sequence;

[0124] The second extraction module 305 is also used to input the fused feature sequence into the transformer model for context feature extraction, and during the extraction process, the multi-head attention mechanism in the transformer model is used to perform attention processing using position weights to obtain the context features of the ECG signal.

[0125] The above-mentioned emotion recognition method, device, computer equipment, storage medium and computer program product based on high-frequency ECG signals obtain an ECG signal sequence composed of QRS wave groups and other sub-waves by segmenting the ECG signal of the target object, thereby following the law of the ECG signal, and therefore can extract pure high-frequency QRS wave group features and other high-frequency sub-wave features from the QRS wave group and other sub-waves, which is beneficial to improving the accuracy of emotional state recognition; in addition, the wavelet coding features of each ECG signal sub-wave, the position coding of each ECG signal sub-wave, the high-frequency QRS wave group features and other high-frequency sub-wave features are fused to obtain a fused feature sequence, thereby enriching the features, which is also beneficial to improving the accuracy of emotional state recognition; moreover, the position weight of each ECG signal sub-wave in the ECG signal sequence is determined, context feature extraction is performed on the fused feature sequence, and attention processing is performed based on the position weight during the extraction process to obtain the context feature of the ECG signal, so that after the attention processing, the context feature can also be ensured to carry position information, so that the emotional state of the target object can be accurately identified according to the context feature, which can effectively improve the recognition accuracy of the emotional state.

[0126] In one embodiment, Figure 4 As shown, the device also includes:

[0127] The emotion guidance module 407 is used to generate prompt words according to the emotional state when the emotional state meets the preset state conditions; input the prompt words into the large language model to generate guidance information for emotion regulation using the large language model.

[0128] In the above embodiment, guidance information is generated according to the emotional state of the target object, so that effective emotion guidance information can be generated using a large language model. Emotional guidance of the target object can help improve or enhance the target object's emotions and enable the target object to achieve effective emotion control.

[0129] In one embodiment, Figure 4 As shown, the device also includes:

[0130] The recommendation module 408 is used to find out multiple music that matches the emotional state from the music library as music to be recommended; obtain basic data and historical listening data of the target object; perform feature extraction on the basic data and historical listening data to obtain interest features of the target object; extract music features of the music to be recommended from relevant data of the music to be recommended; determine the recommendation score of the music to be recommended based on the interest features and music features; select at least one music whose recommendation score meets the preset score conditions as the target music from the music to be recommended; and recommend the target music to the target object.

[0131] In the above embodiment, music is recommended in combination with the emotional state and interest characteristics of the target object. On the one hand, music that the user is interested in can be recommended and the recommended music can be ensured to match the emotional atmosphere of the target object. On the other hand, the playing time and playing rate of the music can be increased.

[0132] Each module in the above-mentioned emotion recognition device based on high-frequency electrocardiogram signals can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0133] In one embodiment, a computer device is provided, which may be a server, and includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as electrocardiogram signals. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, an emotion recognition method based on high-frequency electrocardiogram signals is implemented.

[0134] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an emotion recognition method based on high-frequency electrocardiogram signals is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0135] Those skilled in the art will understand that Figure 5The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0136] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the above-mentioned emotion recognition method based on high-frequency electrocardiogram signals are implemented.

[0137] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned emotion recognition method based on high-frequency electrocardiogram signals are implemented.

[0138] In one embodiment, a computer program product is provided, including a computer program, which implements the steps of the above-mentioned emotion recognition method based on high-frequency electrocardiogram signals when executed by a processor.

[0139] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0140] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0141] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An emotion recognition method based on high-frequency electrocardiogram signals, characterized in that: The method comprises: Segmenting the target object's ECG signal to obtain an ECG signal sequence composed of different ECG signal sub-waves; the different ECG signal sub-waves include QRS complexes and other sub-waves; the other sub-waves include P waves and T waves; Extracting high-frequency features from the QRS complex and the other sub-waves respectively to obtain high-frequency QRS complex features and other high-frequency sub-wave features; the high-frequency QRS complex features include length features, amplitude features, morphological features and distance features between different sub-waves of the high-frequency QRS complex, and the other high-frequency sub-wave features include length features, amplitude features and morphological features of other high-frequency sub-waves; Fusing the wavelet coding features of each of the ECG signal wavelets, the high-frequency QRS complex features, and the other high-frequency wavelet features, and weighting the obtained fusion results using the position coding of each of the ECG signal wavelets to obtain a fusion feature sequence; Determining the position weight of each ECG signal wavelet in the ECG signal sequence; Extracting context features from the fused feature sequence using a transformer model, and performing attention processing based on the position weights during the extraction process to obtain context features of the electrocardiogram signal; The emotional state of the target object is identified according to the contextual features.

2. The method according to claim 1, characterized in that The step of segmenting the target object's ECG signal to obtain an ECG signal sequence composed of different ECG signal wavelets includes: Determine the segmentation points of each sub-wave; Dividing the electrocardiogram signal into a plurality of electrocardiogram signal sub-waves based on each of the segmentation points; The multiple ECG signal wavelets are sorted according to the acquisition time of the ECG signal to obtain an ECG signal sequence composed of different ECG signal wavelets.

3. The method according to claim 1, characterized in that Determining the position weight of each ECG signal wavelet in the ECG signal sequence includes: Determining the relative distance between each ECG signal wavelet in the ECG signal sequence; The position weight of each of the electrocardiogram signal wavelets is determined according to the relative distance.

4. The method according to claim 1, characterized in that: The wavelet coding features of each of the ECG signal wavelets, the high-frequency QRS complex features and the other high-frequency wavelet features are fused, and the obtained fusion results are weighted by using the position coding of each of the ECG signal wavelets to obtain a fusion feature sequence, which includes: Performing encoding processing on each ECG signal wavelet in the ECG signal sequence to obtain a wavelet encoding feature of each ECG signal wavelet; Performing position coding on each ECG signal wavelet in the ECG signal sequence to obtain a position coding of each ECG signal wavelet; Fusing the wavelet coding features of each of the ECG signal wavelets, the high-frequency QRS complex features, and the other high-frequency wavelet features, and weighting the obtained fusion results using the position coding of each of the ECG signal wavelets to obtain a fusion feature sequence; The context features of the electrocardiogram signal obtained by extracting context features of the fused feature sequence through the transformer model and performing attention processing based on the position weight during the extraction process include: The fused feature sequence is input into a transformer model for context feature extraction, and during the extraction process, the position weight is used for attention processing through a multi-head attention mechanism in the transformer model to obtain the context feature of the electrocardiogram signal.

5. The method according to any one of claims 1 to 4, characterized in that: After identifying the emotional state of the target object according to the context feature, the method further includes: When the emotional state meets the preset state condition, generating a prompt word according to the emotional state; The prompt words are input into a large language model to generate guidance information for emotion regulation using the large language model.

6. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Searching a plurality of music pieces that match the emotional state from a music library as music pieces to be recommended; Obtaining basic data and historical listening data of the target object; Extracting features from the basic data and the historical listening data to obtain interest features of the target object; Extracting music features of the music to be recommended from relevant data of the music to be recommended; Determining a recommendation score of the music to be recommended according to the interest feature and the music feature; Among the music to be recommended, at least one music whose recommendation score meets a preset score condition is selected as target music; The target music is recommended to the target object.

7. An emotion recognition device based on high-frequency electrocardiogram signals, characterized in that: The device comprises: A segmentation module is used to segment the ECG signal of the target object to obtain an ECG signal sequence composed of different ECG signal sub-waves; the different ECG signal sub-waves include QRS wave groups and other sub-waves; the other sub-waves include P waves and T waves; A first extraction module is used to extract high-frequency features from the QRS complex and the other sub-waves, respectively, to obtain high-frequency QRS complex features and other high-frequency sub-wave features; the high-frequency QRS complex features include length features, amplitude features, morphological features and distance features between different sub-waves of the high-frequency QRS complex, and the other high-frequency sub-wave features include length features, amplitude features and morphological features of other high-frequency sub-waves; A fusion module, used for fusing the wavelet coding features of each of the ECG signal wavelets, the high-frequency QRS complex features and the other high-frequency wavelet features, and weighting the obtained fusion results by using the position coding of each of the ECG signal wavelets to obtain a fusion feature sequence; A determination module, used to determine the position weight of each ECG signal wavelet in the ECG signal sequence; A second extraction module is used to extract context features from the fused feature sequence through a transformer model, and perform attention processing based on the position weight during the extraction process to obtain context features of the electrocardiogram signal; The recognition module is used to recognize the emotional state of the target object according to the context feature.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images