Emotion recognition method and device based on ballistocardiogram signal

Abstract

The present disclosure provides an emotion recognition method and device based on a ballistocardiogram signal, which can be applied to the technical field of physiological signal processing. The method comprises: obtaining ballistocardiogram signal data of a target object and ballistocardiogram signal data of each of a plurality of candidate objects; performing similarity processing on the ballistocardiogram signal data of the target object and the ballistocardiogram signal data of each of the plurality of candidate objects, determining a candidate object associated with the target object from the plurality of candidate objects, and obtaining at least one reference object; and processing the ballistocardiogram signal data of the reference object by using an emotion recognition model to obtain an emotion recognition result of the target object.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of physiological signal processing, in particular to an emotion recognition method and device based on ballistocardiogram. BACKGROUND

[0002] Emotion recognition is an important research direction in the field of affective computing, and can be applied in multiple fields. Existing emotion recognition technologies generally use external behaviors such as facial expressions, voices, postures, or physiological signals for emotion recognition. Since external behaviors can be controlled by humans, people can hide or disguise according to their own will, and the physiological signals of individuals also differ greatly, therefore, the emotion recognition method in the related art has low recognition accuracy. SUMMARY

[0003] In view of the above problems, the present disclosure provides an emotion recognition method and device based on ballistocardiogram.

[0004] According to a first aspect of the present disclosure, an emotion recognition method based on ballistocardiogram is provided, comprising: obtaining ballistocardiogram data of a target object and ballistocardiogram data of each of a plurality of candidate objects; performing similarity processing on the ballistocardiogram data of the target object and the ballistocardiogram data of each of the plurality of candidate objects, to determine a candidate object associated with the target object from the plurality of candidate objects, to obtain at least one reference object; processing the ballistocardiogram data of the reference object by using an emotion recognition model, to obtain an emotion recognition result of the target object.

[0005] According to an embodiment of the present disclosure, the candidate objects include M, M is a positive integer greater than 1, and the candidate objects are marked with emotion category labels; the similarity processing on the ballistocardiogram data of the target object and the ballistocardiogram data of each of the plurality of candidate objects, to determine a candidate object associated with the target object from the plurality of candidate objects, to obtain at least one reference object, comprises: for each of the two candidate objects in the M candidate objects, determining a first similarity between the two candidate objects by using the ballistocardiogram data and the emotion category labels of the two candidate objects, to obtain a first similarity matrix of MxM dimensions; for each of the M candidate objects, determining a second similarity between the target object and the candidate object by using the ballistocardiogram data of the target object and the ballistocardiogram data of the candidate object, to obtain a second similarity matrix of Mx1 dimensions; performing clustering processing on the M candidate objects and the target object according to the first similarity matrix of MxM dimensions and the second similarity matrix of Mx1 dimensions, to obtain a plurality of clustering clusters; determining a clustering cluster in which the target object is located from the plurality of clustering clusters, to obtain a target cluster; determining the candidate objects contained in the target cluster as the candidate objects associated with the target object, to obtain at least one reference object.

[0006] According to an embodiment of the present disclosure, the heart impact signal data of the target object comprises a plurality of groups of heart impact signal sub-data, and M candidate objects collectively correspond to emotion recognition sub-models, and each candidate object has a corresponding emotion recognition sub-model; determining the second similarity between the target object and the candidate objects by using the heart impact signal data of the candidate objects and the heart impact signal data of the target object comprises: processing the plurality of groups of heart impact signal sub-data by using the emotion recognition sub-models collectively corresponding to the M candidate objects respectively to obtain first emotion category prediction values corresponding to the plurality of groups of heart impact signal sub-data respectively; for the mth candidate object in the M candidate objects, processing the target heart impact signal sub-data by using the emotion recognition sub-model corresponding to the mth candidate object to obtain a second emotion category prediction value, wherein the target heart impact signal sub-data is the heart impact signal sub-data corresponding to the target number of first emotion category prediction values based on a preset ranking rule in the plurality of first emotion category prediction values, 1 m M; determining the second similarity between the target object and the mth candidate object according to the target first emotion category prediction value and the second emotion category prediction value, wherein the target first emotion category prediction value is the maximum first emotion category prediction value in the plurality of first emotion category prediction values.

[0007] According to an embodiment of the present disclosure, determining the first similarity between the two candidate objects by using the heart impact signal data and the emotion category labels of the two candidate objects comprises: for the a th candidate object and the b th candidate object in the M candidate objects, processing the heart impact signal data of the b th candidate object by using the emotion recognition sub-model corresponding to the a th candidate object to obtain a third emotion category prediction value of the b th candidate object, 1 M,1 M; obtaining the first similarity between the a th candidate object and the b th candidate object according to the third emotion category prediction value of the b th candidate object and the emotion category label of the b th candidate object.

[0008] According to an embodiment of the present disclosure, the emotion recognition method based on heart impact signals further comprises: performing feature alignment processing on the heart impact signal data of the target object and the reference object based on a balanced distribution adaptive optimization function to obtain converted signal data of the reference object; and processing the heart impact signal data of the reference object by using the emotion recognition model to obtain the emotion recognition result of the target object comprises: processing the converted signal data of the reference object by using the emotion recognition model to obtain the emotion recognition result of the target object.

[0009] According to an embodiment of the present disclosure, the feature alignment processing is performed on the heart impact signal data of the target object and the reference object based on a balanced distribution adaptive optimization function to obtain the converted signal data of the reference object, including: determining a target edge probability distribution and a target conditional probability distribution according to the heart impact signal data of the target object; determining a reference edge probability distribution and a reference conditional probability distribution according to the heart impact signal data of the reference object; processing the target edge probability distribution, the target conditional probability distribution, the reference edge probability distribution and the reference conditional probability distribution by using the balanced distribution adaptive optimization function to obtain a conversion matrix; and obtaining the converted signal data of the reference object according to the conversion matrix and the heart impact signal data of the reference object.

[0010] According to an embodiment of the present disclosure, the conversion matrix is obtained by processing the target edge probability distribution, the target conditional probability distribution, the reference edge probability distribution and the reference conditional probability distribution by using the balanced distribution adaptive optimization function, including: calculating a difference between the target edge probability distribution and the reference edge probability distribution by using a mean difference function to obtain an edge difference value; calculating a difference between the target conditional probability distribution and the reference conditional probability distribution by using the mean difference function to obtain a conditional difference value; and processing the edge difference value and the conditional difference value by using the balanced distribution adaptive optimization function to obtain the conversion matrix, wherein the balanced distribution adaptive optimization function is used to optimize the distribution difference between the heart impact signal data of the target object and the heart impact signal data of the reference object.

[0011] According to an embodiment of the present disclosure, the heart impact signal data includes at least one of the following: heart rate variability data, respiratory variation data, beat-by-beat heartbeat statistical data, and beat-by-beat heartbeat nonlinear data.

[0012] According to a second aspect of the present disclosure, a method for emotion recognition based on a ballistocardiogram is provided. The method comprises: obtaining ballistocardiogram data of a target object and ballistocardiogram data of each of a plurality of candidate objects; performing similarity processing on the ballistocardiogram data of the target object and the ballistocardiogram data of each of the plurality of candidate objects to determine a candidate object associated with the target object from the plurality of candidate objects, to obtain at least one reference object; and processing the ballistocardiogram data of the reference object by using an emotion recognition model to obtain an emotion recognition result of the target object.

[0013] According to the method and device for emotion recognition based on a ballistocardiogram provided by the present disclosure, the ballistocardiogram data of the target object and the ballistocardiogram data of each of the plurality of candidate objects are subjected to similarity processing to determine a candidate object associated with the target object from the plurality of candidate objects, to obtain at least one reference object; and the ballistocardiogram data of the reference object is processed by using an emotion recognition model to obtain an emotion recognition result of the target object. Since the ballistocardiogram data of the target object and the ballistocardiogram data of each of the plurality of candidate objects are subjected to similarity evaluation, a reference object with high similarity to the target object is determined from the plurality of candidate objects, and the emotion recognition result of the target object is predicted based on the ballistocardiogram data of the reference object, thereby greatly reducing the influence of individual physiological differences on the prediction accuracy of the emotion recognition model, avoiding the introduction of knowledge irrelevant or even conflicting to the target object, and thus overcoming the technical problem of low emotion recognition accuracy caused by physiological differences between individual objects. In addition, selecting a reference object with high similarity for emotion recognition reduces unnecessary computational overhead and helps improve emotion recognition efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0014] The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure with reference to the accompanying drawings.

[0015] Figure 1 A flowchart of a method for emotion recognition based on a ballistocardiogram according to an embodiment of the present disclosure is shown.

[0016] Figure 2 An example schematic diagram of locating a J-peak of ballistocardiogram data according to an embodiment of the present disclosure is shown.

[0017] Figure 3 An example schematic diagram of determining a reference object according to an embodiment of the present disclosure is shown.

[0018] Figure 4 An example schematic diagram of obtaining an emotion recognition result according to an embodiment of the present disclosure is shown.

[0019] Figure 5A structural block diagram of an emotion recognition device based on a ballistocardiogram according to an embodiment of the disclosure is shown. DETAILED DESCRIPTION

[0020] Hereinafter, embodiments according to the disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary, and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. However, it will be apparent to one skilled in the art that one or more embodiments can be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques have been omitted to avoid unnecessarily obscuring the concept of the disclosure.

[0021] The terms used herein are merely used to describe specific embodiments, and are not intended to limit the disclosure. The terms "include", "comprise" and the like used herein indicate the presence of the features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0022] All terms used herein (including technical and scientific terms) have meanings commonly understood by one of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings consistent with the context of the specification, and should not be interpreted in an idealized or overly formal manner.

[0023] In the case of using expressions similar to "at least one of A, B, and C, etc.", in general, it should be interpreted as having a meaning that includes one or more of the corresponding categories, unless otherwise defined (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, a system having B alone, a system having C alone, a system having A and B together, a system having A and C together, a system having B and C together, and / or a system having A, B, and C together, etc.).

[0024] In the process of implementing the disclosure, it is found that, based on the ballistocardiogram collected by the non-contact acquisition device, due to the different personality types between individuals, there are differences in emotions between individuals, and due to the nonlinear time-varying characteristics of the ballistocardiogram itself, there are also individual differences, the emotional characteristics between individuals no longer conform to the independent and identically distributed assumption of existing machine learning, the generalization of the model will be affected, resulting in a significant decline in the accuracy of the model trained on the sample source individuals on the target individuals.

[0025] Therefore, embodiments according to the present disclosure provide a method and device for emotion recognition based on a ballistocardiogram. The method comprises: obtaining ballistocardiogram data of a target object and ballistocardiogram data of each of a plurality of candidate objects; performing similarity processing on the ballistocardiogram data of the target object and the ballistocardiogram data of each of the plurality of candidate objects to determine a candidate object associated with the target object from the plurality of candidate objects, to obtain at least one reference object; and processing the ballistocardiogram data of the reference object by using an emotion recognition model to obtain an emotion recognition result of the target object.

[0026] In the technical solution of the present disclosure, the user information (including but not limited to user personal information, user image information, user device information such as location information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards, necessary security measures are taken, do not violate public order and good customs, and corresponding operation portals are provided for the user to choose authorization or refusal.

[0027] It should be noted that the serial numbers of the operations in the following method are only used to represent the operations for description, and should not be regarded as representing the execution order of the operations. Unless explicitly stated, the method does not need to be executed in the order shown.

[0028] Figure 1 A flowchart of a method for emotion recognition based on a ballistocardiogram according to an embodiment of the present disclosure is shown.

[0029] As shown in Figure 1 , the method comprises operation S110 to operation S130.

[0030] In operation S110, ballistocardiogram data of a target object and ballistocardiogram data of each of a plurality of candidate objects are obtained.

[0031] According to an embodiment of the present disclosure, the target object is an object to be recognized for emotion, and the candidate object can be an object with ballistocardiogram data obtained from a shared database platform.

[0032] According to an embodiment of the present disclosure, the ballistocardiogram data is a physiological signal (Ballistocardiogram, BCG) collected by a non-contact sensor. For example, the sensor is placed in a seat cushion, on the surface, inside or bottom of a mattress, and the ballistocardiogram data is collected without contact.

[0033] In operation S120, similarity processing is performed on the heart impulse signal data of the target object and the heart impulse signal data of each of the plurality of candidate objects, to determine a candidate object associated with the target object from the plurality of candidate objects, to obtain at least one reference object.

[0034] According to an embodiment of the present disclosure, the heart impulse signal data of the target object and the heart impulse signal data of each of the plurality of candidate objects are processed by using a similarity algorithm, to perform similarity evaluation on the target object and the candidate objects in the physiological feature direction, and to select a reference object with high similarity from the plurality of candidate objects.

[0035] According to an embodiment of the present disclosure, the heart impulse signal data of the target object and the heart impulse signal data of the candidate objects have the same feature space and label space, and the data distribution is different.

[0036] According to an embodiment of the present disclosure, the reference object is an object with significant correlation with the target object, for example, the heart impulse signal data of the target object and the candidate objects are subjected to similarity evaluation, and at least one reference object with a high similarity value is selected from the plurality of candidate objects.

[0037] In operation S130, the heart impulse signal data of the reference object is processed by using an emotion recognition model, to obtain an emotion recognition result of the target object.

[0038] According to an embodiment of the present disclosure, the emotion recognition model can be constructed by using a machine learning algorithm based on a classification and regression task (Support Vector Machine (SVM) of Radial Basis Function (RBF) kernel).

[0039] According to an embodiment of the present disclosure, the heart impulse signal data of the reference object is input into the emotion recognition model, and an emotion recognition result is output. The emotion recognition result represents an emotion category prediction result of the target object.

[0040] For example, the emotion recognition result can be a positive result, a negative result, or a neutral result.

[0041] According to an embodiment of the present disclosure, since the similarity of the target object and the heart impulse signal data of each of the plurality of candidate objects is evaluated, the reference object with high similarity to the target object is determined from the plurality of candidate objects, and the emotion recognition result of the target object is predicted based on the heart impulse signal data of the reference object, the generalization ability of the emotion recognition model is improved, irrelevant or even conflicting knowledge can be avoided, the influence of individual physiological differences on the prediction accuracy of the emotion recognition model is greatly reduced, irrelevant or even conflicting knowledge is avoided, and the technical problem of low emotion recognition accuracy caused by physiological differences between individual objects is overcome. In addition, selecting a reference object with high similarity for emotion recognition reduces unnecessary computational overhead and helps improve recognition efficiency.

[0042] According to an embodiment of the present disclosure, the heart impulse signal data includes at least one of heart rate variability data, respiratory variation data, interbeat interval statistics data, and interbeat interval non-linear data.

[0043] According to an embodiment of the present disclosure, the heart rate variability data represents time domain, frequency domain, and non-linear domain features of the heartbeat interval, and the respiratory variation data represents frequency and waveform features of the respiration.

[0044] According to an embodiment of the present disclosure, the interbeat interval (IBI) is the time interval between two consecutive heartbeats, and the interbeat interval statistics data can be statistical data such as the mean, standard deviation, maximum value, and minimum value of the physiological data in this time interval. The interbeat interval non-linear data can be entropy feature data of the physiological data in this time interval.

[0045] Figure 2 An example diagram of locating the J-peak of the heart impulse signal data according to an embodiment of the present disclosure is shown.

[0046] As shown in Figure 2 , the J-peak of the heart impulse signal data is located based on a template matching algorithm. The J-peak is the main peak of the signal occurring during ventricular contraction and blood flow impact on the descending aorta. The interbeat interval (IBI) of the heart impulse signal data is obtained by extracting the interbeat interval. The interval sequence data of the extracted interbeat interval is divided into sub-sequences each containing 100 IBIs, and the heart rate variability data, respiratory variation data, interbeat interval statistics data, and interbeat interval non-linear data are extracted from each sub-sequence.

[0047] According to an embodiment of the present disclosure, the preprocessing of the heart impulse signal data of the target object and the candidate objects includes: obtaining reference heart impulse signal data of the target object and the candidate objects in a calm heartbeat state, and obtaining mean values of the heart rate variability features, the respiratory variation features, the interbeat interval statistics features, and the interbeat interval non-linear features, respectively.

[0048] According to an embodiment of the present disclosure, the feature data of the target object and the candidate object is normalized to [-1, 1] for the mean value of each feature.

[0049] For example, the reference heart rate variability data of the target object and the candidate object in a calm heartbeat state is obtained, the mean value corresponding to the heart rate variability feature is obtained, the heart rate variability data of the target object and the candidate object is standardized based on the mean value, and the preprocessed heart rate variability data of the target object and the candidate object is obtained.

[0050] According to an embodiment of the present disclosure, the candidate objects include M, M is a positive integer greater than 1, and the candidate objects are labeled with emotion category labels; the similarity between the target object and the candidate objects is processed based on the PPG data of the target object and the PPG data of the candidate objects, the candidate objects associated with the target object are determined from the plurality of candidate objects, and at least one reference object is obtained by: for each of the M candidate objects, the first similarity between the two candidate objects is determined based on the PPG data of the two candidate objects and the emotion category labels, and a MxM dimension first similarity matrix is obtained; for each of the M candidate objects, the second similarity between the target object and the candidate object is determined based on the PPG data of the candidate object and the PPG data of the target object, and a Mx1 dimension second similarity matrix is obtained; the M candidate objects and the target object are clustered based on the MxM dimension first similarity matrix and the Mx1 dimension second similarity matrix, and a plurality of clustering clusters are obtained; the clustering cluster in which the target object is located is determined from the plurality of clustering clusters, and a target cluster is obtained; the candidate objects contained in the target cluster are determined as the candidate objects associated with the target object, and at least one reference object is obtained.

[0051] According to an embodiment of the present disclosure, the target object has no emotion category label, and each candidate object is labeled with an emotion category label, for example, the first candidate object is labeled with an emotion category label 0, indicating that the candidate object 1 is of a negative category.

[0052] According to an embodiment of the present disclosure, the first similarity between the two candidate objects is obtained by processing the PPG data of the two candidate objects and the emotion category labels of the two candidate objects using a label similarity algorithm.

[0053] According to an embodiment of the present disclosure, each candidate object needs to calculate the first similarity with itself and the remaining M-1 candidate objects respectively, and then obtain the Mx1 dimension first similarity matrix corresponding to each candidate object, and thus the MxM dimension first similarity matrix of the M candidate objects is obtained.

[0054] According to an embodiment of the present disclosure, the second similarity between the target object and each candidate object is calculated by using a label similarity algorithm, and a second similarity matrix of M*1 dimension is obtained.

[0055] According to an embodiment of the present disclosure, the first similarity matrix of M*M dimension and the second similarity matrix of M*1 dimension constitute a similarity matrix of M*(M+1) dimension, the similarity matrix of M*(M+1) dimension is taken as a feature of topic clustering, and a clustering algorithm (Kmeans) is used to perform clustering processing on the M candidate objects and the target object, and a plurality of clustering results are obtained.

[0056] For example, the plurality of clustering results can include a clustering result corresponding to a clustering number of 1, a clustering result corresponding to a clustering number of 2, and a clustering result corresponding to a clustering number of 3. The variance ratio values of the plurality of clustering results are sorted, the clustering result corresponding to the maximum variance ratio value is determined as an optimal clustering result, and the optimal clustering result includes a plurality of clustering clusters.

[0057] In an embodiment, the clustering algorithm Kmeans is shown in formula (1):

[0058] (1);

[0059] wherein, the clustering result corresponding to the clustering number of 1 is represented by, the clustering result corresponding to the clustering number of K is represented by, the similarity matrix of M*(M+1) dimension is represented by.

[0060] In an embodiment, the variance ratio value is shown in formula (2):

[0061] (2);

[0062] wherein, the variance ratio value corresponding to the clustering result corresponding to the clustering number of K is represented by, the trace of the matrix is represented by, the covariance matrix between the plurality of clustering clusters in the clustering result is represented by, the covariance matrix of the data in the clustering result is represented by, the number of the target object and the candidate object is represented by.

[0063] According to an embodiment of the present disclosure, the optimal clustering result includes a plurality of clustering clusters, a clustering cluster in which the target object is located is determined as a target cluster, candidate objects included in the target cluster are determined as candidate objects associated with the target object, and the candidate objects associated with the target object are determined as reference objects.

[0064] According to an embodiment of the present disclosure, by defining the similarity between candidate objects and between the candidate objects and the target object, clustering all objects according to the similarity matrix and then ranking, at least one reference object similar to the target object with high confidence is adaptively selected into the final object set used for recognition, which greatly reduces the influence of individual physiological differences on the prediction accuracy of the emotion recognition model.

[0065] According to an embodiment of the present disclosure, the cardiac impulse signal data of the target object includes multiple groups of cardiac impulse signal sub-data, and M candidate objects correspond to emotion recognition sub-models, and each candidate object has a corresponding emotion recognition sub-model; determining the second similarity between the target object and the candidate objects by using the cardiac impulse signal data of the target object and the cardiac impulse signal data of the candidate objects can include: calculating the second similarity between the target object and each candidate object by using a label similarity algorithm to obtain a second similarity matrix of Mx1. Specifically, the emotion recognition sub-models corresponding to the M candidate objects are used to process the multiple groups of cardiac impulse signal sub-data respectively to obtain first emotion category prediction values corresponding to the multiple groups of cardiac impulse signal sub-data respectively; for the mth candidate object in the M candidate objects, the emotion recognition sub-model corresponding to the mth candidate object is used to process the target cardiac impulse signal sub-data to obtain a second emotion category prediction value, wherein the target cardiac impulse signal sub-data is the cardiac impulse signal sub-data corresponding to the target number of first emotion category prediction values in the multiple first emotion category prediction values determined based on a preset ranking rule, 1 m M; according to the target first emotion category prediction value and the second emotion category prediction value, the second similarity between the target object and the mth candidate object is determined, wherein the target first emotion category prediction value is the maximum first emotion category prediction value in the multiple first emotion category prediction values.

[0066] According to an embodiment of the present disclosure, the multiple first emotion category prediction values are probability values of predicting that each group of cardiac impulse signal sub-data is a certain emotion category.

[0067] For example, the target number is p, and there are 3 emotion categories, the first emotion category prediction values corresponding to each emotion category are sorted in ascending order, the top p / 3 larger first emotion category prediction values are determined from the multiple first emotion category prediction values corresponding to each emotion category, thereby obtaining p larger first emotion category prediction values, and the cardiac impulse signal sub-data corresponding to the p larger first emotion category prediction values are determined as the target cardiac impulse signal sub-data.

[0068] According to an embodiment of the present disclosure, the physiological signal of the target object is sampled multiple times to obtain multiple groups of cardiac impulse signal sub-data. The emotion recognition sub-model can be a classifier model.

[0069] According to an embodiment of the present disclosure, the target object does not have a real emotion category label, and since the label confidence obtained by predicting the target object based on the candidate object trained emotion recognition sub-model is not high, a high confidence pseudo label selection strategy is designed to select a high confidence pseudo label corresponding to the target object.

[0070] According to an embodiment of the present disclosure, an emotion recognition sub-model is trained jointly using the M candidate objects' heart shock signal data and emotion category labels, and each set of target object heart shock signal sub-data is processed based on the joint emotion recognition sub-model to obtain a first emotion category prediction value, which is a pseudo emotion category label of the target object, representing the probability that a single set of heart shock signal sub-data belongs to a certain emotion category.

[0071] According to an embodiment of the present disclosure, the maximum first emotion category prediction value is selected from a plurality of first emotion category prediction values, and the maximum first emotion category prediction value is determined as the target first emotion category prediction value, and the category corresponding to the target first emotion category prediction value is the pseudo emotion category label.

[0072] According to an embodiment of the present disclosure, a second similarity between the target object and the mth candidate object is calculated: first, the target heart shock signal sub-data is processed using the emotion recognition sub-model corresponding to the mth candidate object to obtain a second emotion category prediction value, and then the second similarity between the target object and the mth candidate object is determined according to the target first emotion category prediction value and the second emotion category prediction value.

[0073] In an embodiment, the second similarity As shown in formula (3):

[0074] (3);

[0075] Wherein, represents the mth candidate object, represents the target object, represents the target first emotion category prediction value, represents the second emotion category prediction value, represents the target heart shock signal sub-data, represents the emotion recognition sub-model of the mth candidate object, represents the label similarity evaluation algorithm.

[0076] According to an embodiment of the present disclosure, a high confidence pseudo label selection strategy is designed to select the target heart shock signal sub-data corresponding to the credible pseudo label, and then the target object and the candidate object are evaluated based on the label similarity, providing high confidence support for subsequent screening of associated reference objects.

[0077] According to an embodiment of the present disclosure, determining the first similarity between the two candidate objects by using the heart shock signal data and the emotion category label of each of the two candidate objects can comprise: processing the heart shock signal data and the emotion category label of each of the two candidate objects by using a label similarity algorithm to obtain the first similarity between the two candidate objects. Specifically, for the a th candidate object and the b th candidate object in the M candidate objects, processing the heart shock signal data of the b th candidate object by using the emotion recognition sub-model corresponding to the a th candidate object to obtain a third emotion category prediction value of the b th candidate object, 1 M, 1 M; obtaining the first similarity between the a th candidate object and the b th candidate object according to the third emotion category prediction value of the b th candidate object and the emotion category label of the b th candidate object.

[0078] According to an embodiment of the present disclosure, the emotion category label of the b th candidate object is the real label emotion category of the b th candidate object.

[0079] In an embodiment, the first similarity As shown in formula (4):

[0080] (4) ;

[0081] wherein, representing the a th candidate object, representing the b th candidate object, representing the emotion category label of the b th candidate object, representing the third emotion category prediction value of the b th candidate object, representing the heart shock signal data of the b th candidate object, representing the emotion recognition sub-model of the a th candidate object.

[0082] Figure 3 An example schematic diagram for determining a reference object according to an embodiment of the present disclosure is shown.

[0083] As Figure 3 shown, the candidate object set includes M candidate objects (candidate object 1, candidate object 2, …, candidate object M), and similarity evaluation is performed based on a label similarity evaluation algorithm, and each candidate corresponds to a Mx1-dimensional first similarity matrix 1 ( ) corresponding to each candidate; candidate object 2 corresponds to a Mx1-dimensional first similarity matrix 2 ( ) corresponding to each candidate; candidate object M corresponds to a Mx1-dimensional first similarity matrix M ( ) corresponding to each candidate; the target object corresponds to a Mx1-dimensional second similarity matrix ( ), so M candidate objects obtain a first similarity matrix of MxM dimensions, and a second similarity matrix of Mx1 dimensions is obtained according to the first similarity matrix of MxM dimensions and the second similarity matrix of Mx1 dimensions, to form a similarity matrix of Mx(M+1) dimensions . The similarity matrix of Mx(M+1) dimensions As a feature of the subject clustering, the clustering algorithm is used to perform clustering processing on the M candidate objects and the target object, to obtain different clustering results (Mx1 dimensions) obtained by different clustering numbers The variance ratio value of the different clustering results is used to select q candidate objects with significant correlation with the target object from the candidate object set, to form a reference object set.

[0084] According to an embodiment of the present disclosure, the emotion recognition method based on the ballistocardiogram further includes: performing feature alignment processing on the ballistocardiogram data of the target object and the reference object based on a balanced distribution adaptive optimization function, to obtain converted signal data of the reference object; processing the ballistocardiogram data of the reference object by using the emotion recognition model to obtain the emotion recognition result of the target object, including: processing the converted signal data of the reference object by using the emotion recognition model to obtain the emotion recognition result of the target object.

[0085] According to an embodiment of the present disclosure, the ballistocardiogram data of the target object and the ballistocardiogram data of the reference object have the same feature space and label space, and the data distribution is different, so the features of the ballistocardiogram data of the target object and the reference object are aligned by using a balanced distribution adaptive optimization function (Balanced Distribution Adaptation, BDA).

[0086] According to an embodiment of the present disclosure, the converted signal data of the reference object is the ballistocardiogram data after the feature alignment processing, and the converted signal data of the reference object is processed by using the emotion recognition model to obtain the emotion recognition result of the target object.

[0087] According to an embodiment of the present disclosure, the feature alignment processing on the ballistocardiogram data of the target object and the reference object based on the balanced distribution adaptive optimization function includes: determining a target edge probability distribution and a target conditional probability distribution according to the ballistocardiogram data of the target object; determining a reference edge probability distribution and a reference conditional probability distribution according to the ballistocardiogram data of the reference object; processing the target edge probability distribution, the target conditional probability distribution, the reference edge probability distribution, and the reference conditional probability distribution by using the balanced distribution adaptive optimization function to obtain a conversion matrix; and obtaining the converted signal data of the reference object according to the conversion matrix and the ballistocardiogram data of the reference object.

[0088] According to an embodiment of the present disclosure, the target object and the reference object correspond to a plurality of groups of ballistocardiogram sub-data respectively, the target marginal probability distribution is an overall probability distribution of the plurality of groups of ballistocardiogram sub-data without considering the emotion category label, and the target conditional probability distribution is a probability distribution of the plurality of groups of ballistocardiogram sub-data given a certain emotion category label.

[0089] According to an embodiment of the present disclosure, the optimization objective of the balanced distribution adaptive optimization function is to simultaneously reduce the marginal probability distribution and the conditional probability distribution between the target object and the reference object by a certain weight, and to minimize the distribution difference.

[0090] According to an embodiment of the present disclosure, the conversion matrix is obtained by solving in the process of achieving the optimization objective of the balanced distribution adaptive optimization function; and the conversion signal data of the reference object is obtained by multiplying the transposed matrix of the conversion matrix and the ballistocardiogram data of the reference object.

[0091] In an embodiment, the conversion signal data of the target object is obtained by multiplying the transposed matrix of the conversion matrix and the ballistocardiogram data of the target object. As shown in formula (5):

[0092] (5);

[0093] wherein, the transposed matrix of the conversion matrix, the ballistocardiogram data of the reference object set.

[0094] According to an embodiment of the present disclosure, the conversion signal data of the target object can be obtained by multiplying the transposed matrix of the conversion matrix and the ballistocardiogram data of the target object. The two probability distributions between the conversion signal data of the target object and the conversion signal data of the reference object are aligned.

[0095] According to an embodiment of the present disclosure, the feature alignment processing is divided into two parts, one is to align the feature distribution of the target object and the reference object given a certain emotion category label, and the other is to align the overall feature distribution of the target object and the reference object without considering the emotion category label. The emotion recognition is performed on the basis of the aligned features, and the generalization of the emotion recognition model is improved.

[0096] ​​​According to an embodiment of the present disclosure, the processing of the target marginal probability distribution, the target conditional probability distribution, the reference marginal probability distribution and the reference conditional probability distribution by the balanced distribution adaptive optimization function to obtain the conversion matrix comprises: calculating the difference between the target marginal probability distribution and the reference marginal probability distribution by a mean difference function to obtain an edge difference value; calculating the difference between the target conditional probability distribution and the reference conditional probability distribution by the mean difference function to obtain a conditional difference value; and processing the edge difference value and the conditional difference value by the balanced distribution adaptive optimization function to obtain the conversion matrix, wherein the balanced distribution adaptive optimization function is used to optimize the distribution difference between the heart shock signal data of the target object and the reference object.

[0097] According to an embodiment of the present disclosure, the mean difference function can be a maximum mean discrepancy (MMD).

[0098] In an embodiment, the balanced distribution adaptive optimization function As shown in formula (6):

[0099] (6);

[0100] wherein, the balance factor is represented by, the target marginal probability distribution is represented by, the reference marginal probability distribution is represented by, the target conditional probability distribution is represented by, the reference conditional probability distribution is represented by, the reference object set selected from the candidate object set is represented by, the target object is represented by D the maximum mean discrepancy function is represented by D the edge difference value is represented by D the conditional difference value is represented by, the single group of heart shock signal sub-data of the reference object is represented by, the single group of heart shock signal sub-data of the target object is represented by.

[0101] According to an embodiment of the present disclosure, since the target object has no real emotion class label, cannot be calculated, and the class conditional probability is used to replace in formula (6) .

[0102] According to an embodiment of the present disclosure, when , the main role of the marginal probability distribution in domain adaptation is reduced, and when , the main role of the conditional probability distribution in domain adaptation is reduced.

[0103] According to an embodiment of the present disclosure, the balanced distribution adaptive optimization function such as formula (6) is expanded to obtain formula (7):

[0104] (7);

[0105] Wherein, ns represents the total number of groups of the reference object set, that is, the total sampling times, nt represents the total number of groups of the target object, C represents the emotion category, represents the i-th group of heart impact signal sub-data of the reference object, represents the j-th group of heart impact signal sub-data of the target object, represents the number of groups of the heart impact signal sub-data of the reference object set belonging to the c-th emotion category label, represents the number of groups of the heart impact signal sub-data of the target object belonging to the c-th emotion category label, represents the sample set belonging to the c-th emotion category label in the reference object set, represents the sample set belonging to the c-th emotion category label in the target object.

[0106] According to an embodiment of the present disclosure, formula (7) is subjected to matrix transformation and regularization transformation to obtain formula (8):

[0107] (8);

[0108] Wherein, X represents , a matrix composed of, represents a regularization parameter, represents a conversion matrix, represents an identity matrix, represents a matrix norm (Frobenius norm), , 1 is a matrix with elements of 1, , are all maximum mean difference matrices, wherein, , , represents the i-th group of heart impact signal sub-data, represents the j-th group of heart impact signal sub-data.

[0109] According to an embodiment of the present disclosure, formula (8) is subjected to Lagrange function transformation to obtain formula (9):

[0110] (9);

[0111] Wherein, represents a Lagrange operator, .

[0112] According to an embodiment of the present disclosure, formula (9) is solved to obtain the optimal conversion matrix A.

[0113] (10);

[0114] wherein the optimal conversion matrix A is d smallest eigenvectors of formula (10).

[0115] Figure 4 An example schematic diagram of obtaining an emotion recognition result is shown according to an embodiment of the present disclosure.

[0116] As Figure 4 shown, the target object's PPG data and the PPG data of each of the plurality of candidate objects are respectively preprocessed, the preprocessing including feature extraction and standardization processing, and then the PPG data of the target object and the PPG data of each of the plurality of candidate objects are processed for similarity based on a label similarity evaluation algorithm, and then a candidate object associated with the target object is determined from the plurality of candidate objects based on a clustering selection algorithm to obtain at least one reference object; the PPG data of the target object and the reference object are processed for feature alignment based on a balanced distribution adaptive optimization function to obtain conversion signal data of the reference object; and the conversion signal data of the reference object is processed using an emotion recognition model to obtain an emotion recognition result of the target object.

[0117] According to an embodiment of the present disclosure, the emotion recognition model includes N binary classifiers, N is a positive integer, and the number of binary classifiers is determined based on the number of sample emotion categories. The emotion recognition model is trained based on the following operations: obtaining training samples, the training samples including conversion signal data of sample reference objects and sample emotion category labels of sample target objects, the sample reference objects being objects associated with the sample target objects; performing feature extraction on the conversion signal data of the sample reference objects to obtain signal features; processing the signal features using the N binary classifiers to obtain sample emotion category prediction values corresponding to each binary classifier; performing statistics on the N sample prediction values based on a statistical function to obtain prediction scores corresponding to each sample emotion category; determining an emotion category corresponding to the maximum prediction score among the plurality of prediction scores as the sample emotion recognition result, wherein the sample emotion recognition result represents a predicted result of the emotion category of the sample target object; and training an initial emotion recognition model based on the sample emotion recognition result and the sample emotion category labels to obtain the emotion recognition model.

[0118] According to an embodiment of the present disclosure, the number of binary classifiers is determined based on permutation and combination of the number of sample emotion categories, for example, if the sample emotion categories include negative, positive, and neutral categories, three binary classifiers are needed to be constructed, the first binary classifier is constructed based on the negative and positive categories, the second binary classifier is constructed based on the negative and neutral categories, and the third binary classifier is constructed based on the neutral and positive categories.

[0119] According to an embodiment of the present disclosure, the number of sample reference objects needs to be selected according to the training sample situation, and in the present scheme, the highest accuracy is obtained when the number of sample reference objects is set to 9.

[0120] According to an embodiment of the present disclosure, the converted signal data of the sample reference object is the ballistocardiogram after the sample reference object and the sample target object are subjected to feature alignment processing.

[0121] In an embodiment, a classifier is constructed for the u-th emotion category and the v-th emotion category As shown in (11):

[0122] (11) ;

[0123] wherein, denotes the predicted value of the sample emotion category, denotes the signal feature, the value of 1 represents the predicted score of the u-th emotion category plus 1, the value of -1 represents the predicted score of the v-th emotion category plus 1, all denote the classifier parameters, .

[0124] In an embodiment, the predicted score corresponding to the v-th sample emotion category is As shown in (12):

[0125] (12) ;

[0126] wherein, denotes the indicator function.

[0127] According to an embodiment of the present disclosure, the emotion category corresponding to the maximum predicted score in the plurality of predicted scores is determined as the sample emotion recognition result, the loss value between the sample emotion recognition result and the sample emotion category label can be calculated by using the loss function, the parameters of the initial emotion recognition model are adjusted according to the loss value, and the emotion recognition model is obtained.

[0128] According to an embodiment of the present disclosure, the verification method of the recognition effect of the emotion recognition model is a leave-one-subject-out (LOSO) method, that is, the data of one subject is left as a test set, and the data of the remaining subjects is used as a training set, until all subjects are tested. The evaluation index can be accuracy, precision, recall, etc.

[0129] According to an embodiment of the present disclosure, the initial emotion recognition model is trained based on the conversion signal data of the sample reference object, and the initial emotion recognition model is adjusted according to the sample emotion category label and the sample emotion recognition result of the sample target object to obtain an optimized emotion recognition model, which improves the accuracy and generalization of the model in recognizing the emotion of the target object, and avoids introducing knowledge irrelevant or even conflicting to the target object, thereby reducing the risk of negative transfer.

[0130] Based on the above-mentioned emotion recognition method based on the ballistocardiogram, the present disclosure further provides an emotion recognition device based on the ballistocardiogram. The following will be described in combination with Figure 5 The device will be described in detail.

[0131] Figure 5 The structure block diagram of the emotion recognition device based on the ballistocardiogram according to an embodiment of the present disclosure is shown.

[0132] As Figure 5 shown, the emotion recognition device based on the ballistocardiogram 500 of the embodiment includes an enhancement module 510, an extraction module 520, and an identification module 530.

[0133] The acquisition module 510 is configured to acquire the ballistocardiogram data of the target object and the ballistocardiogram data of each of the plurality of candidate objects. In an embodiment, the acquisition module 510 can be configured to perform the operation S110 described above, and details are not repeated here.

[0134] The screening module 520 is configured to perform similarity processing on the ballistocardiogram data of the target object and the ballistocardiogram data of each of the plurality of candidate objects, determine the candidate object associated with the target object from the plurality of candidate objects, and obtain at least one reference object. In an embodiment, the screening module 520 can be configured to perform the operation S120 described above, and details are not repeated here.

[0135] The identification module 530 is configured to process the ballistocardiogram data of the reference object by using the emotion recognition model to obtain an emotion recognition result of the target object. In an embodiment, the identification module 530 can be configured to perform the operation S130 described above, and details are not repeated here.

[0136] According to an embodiment of the present disclosure, the screening module 520 comprises a first screening sub-module, a second screening sub-module, a third screening sub-module, a fourth screening sub-module, and a fifth screening sub-module.

[0137] The first screening sub-module is configured to determine, for each of the two candidate objects in the M candidate objects, a first similarity between the two candidate objects by using the respective heart shock signal data and the emotion category label of the two candidate objects, to obtain a first similarity matrix of MxM dimensions.

[0138] The second screening sub-module is configured to determine, for each of the M candidate objects, a second similarity between the target object and the candidate object by using the heart shock signal data of the target object and the candidate object, to obtain a second similarity matrix of Mx1 dimensions.

[0139] The third screening sub-module is configured to perform clustering processing on the M candidate objects and the target object according to the first similarity matrix of MxM dimensions and the second similarity matrix of Mx1 dimensions, to obtain a plurality of clustering clusters.

[0140] The fourth screening sub-module is configured to determine, from the plurality of clustering clusters, a clustering cluster in which the target object is located, to obtain a target cluster.

[0141] The fifth screening sub-module is configured to determine the candidate objects contained in the target cluster as the candidate objects associated with the target object, to obtain at least one reference object.

[0142] According to an embodiment of the present disclosure, the second screening sub-module comprises a first screening unit, a second screening unit, and a third screening unit.

[0143] The first screening unit is configured to process a plurality of groups of heart shock signal sub-data by using an emotion recognition sub-model commonly corresponding to the M candidate objects, to obtain a first emotion category prediction value corresponding to each of the plurality of groups of heart shock signal sub-data.

[0144] The second screening unit is configured to process, for an mth candidate object in the M candidate objects, target heart shock signal sub-data by using an emotion recognition sub-model corresponding to the mth candidate object, to obtain a second emotion category prediction value, wherein the target heart shock signal sub-data is heart shock signal sub-data corresponding to a target number of first emotion category prediction values in the plurality of first emotion category prediction values based on a preset ranking rule, 1 m M.

[0145] The third screening unit is configured to determine a second similarity between the target object and the mth candidate object according to a target first emotion category prediction value and the second emotion category prediction value, wherein the target first emotion category prediction value is a maximum first emotion category prediction value in the plurality of first emotion category prediction values.

[0146] According to an embodiment of the present disclosure, the first screening submodule comprises a fourth screening unit and a fifth screening unit.

[0147] The fourth screening unit is configured to, for the a th candidate object and the b th candidate object in the M candidate objects, process the b th candidate object's ballistocardiogram data by using the a th candidate object's corresponding emotion recognition submodel to obtain a third emotion category prediction value of the b th candidate object.

[0148] The fifth screening unit is configured to obtain a first similarity between the a th candidate object and the b th candidate object according to the third emotion category prediction value of the b th candidate object and an emotion category label of the b th candidate object.

[0149] According to an embodiment of the present disclosure, the emotion recognition device 500 based on ballistocardiogram further comprises an optimization module.

[0150] The optimization module is configured to perform feature alignment processing on the ballistocardiogram data of the target object and the reference object based on a balanced distribution adaptive optimization function to obtain converted signal data of the reference object.

[0151] According to an embodiment of the present disclosure, the optimization module comprises a first optimization submodule, a second optimization submodule, a third optimization submodule and a fourth optimization submodule.

[0152] The first optimization submodule is configured to determine a target edge probability distribution and a target conditional probability distribution according to the ballistocardiogram data of the target object.

[0153] The second optimization submodule is configured to determine a reference edge probability distribution and a reference conditional probability distribution according to the ballistocardiogram data of the reference object.

[0154] The third optimization submodule is configured to process the target edge probability distribution, the target conditional probability distribution, the reference edge probability distribution and the reference conditional probability distribution by using the balanced distribution adaptive optimization function to obtain a conversion matrix.

[0155] The fourth optimization submodule is configured to obtain the converted signal data of the reference object according to the conversion matrix and the ballistocardiogram data of the reference object.

[0156] According to an embodiment of the present disclosure, the third optimization submodule comprises a first optimization unit, a second optimization unit and a third optimization unit.

[0157] The first optimization unit is configured to calculate the difference between the target edge probability distribution and the reference edge probability distribution by using a mean difference function to obtain an edge difference value.

[0158] The second optimization unit is configured to calculate a difference between the target conditional probability distribution and the reference conditional probability distribution by using a mean difference function, to obtain a conditional difference value.

[0159] The third optimization unit is configured to process the edge difference value and the conditional difference value by using a balanced distribution adaptive optimization function, to obtain the conversion matrix, wherein the balanced distribution adaptive optimization function is used to optimize a distribution difference between the target object and the reference object.

[0160] According to an embodiment of the present disclosure, any of the modules, sub-modules, units, and sub-units can be combined in one module, or any of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of the modules can be combined with at least part of the functions of other modules, and implemented in one module. According to an embodiment of the present disclosure, at least one of the modules, sub-modules, units, and sub-units can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable manner of hardware or firmware by integrating or packaging the circuit, or implemented in any one of software, hardware, and firmware or in a proper combination of any of the above. Alternatively, at least one of the modules, sub-modules, units, and sub-units can be at least partially implemented as a computer program module that can perform corresponding functions when the computer program module is run.

[0161] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways, even if such combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments of this disclosure may be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0162] The embodiments according to this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for emotion recognition based on a heart impact signal, characterized in that, The method comprises: obtaining heart impact signal data of a target object and heart impact signal data of each of a plurality of candidate objects; performing similarity processing on the heart impact signal data of the target object and the heart impact signal data of each of the plurality of candidate objects to determine a candidate object associated with the target object from the plurality of candidate objects, to obtain at least one reference object; performing feature alignment processing on the heart impact signal data of the target object and the reference object based on a balanced distribution adaptive optimization function, to obtain converted signal data of the reference object; processing the heart impact signal data of the reference object using an emotion recognition model to obtain an emotion recognition result of the target object, comprising: processing the converted signal data of the reference object using an emotion recognition model to obtain an emotion recognition result of the target object.

2. The method of claim 1, wherein, The plurality of candidate objects comprises M, M being a positive integer greater than 1, and the plurality of candidate objects are labeled with emotion category labels; performing similarity processing on the heart impact signal data of the target object and the heart impact signal data of each of the plurality of candidate objects to determine a candidate object associated with the target object from the plurality of candidate objects, to obtain at least one reference object comprises: for each of the M candidate objects, determining a first similarity between the two candidate objects using the heart impact signal data and the emotion category labels of the two candidate objects, to obtain a first similarity matrix of MxM dimensions; for each of the M candidate objects, determining a second similarity between the target object and the candidate object using the heart impact signal data of the target object and the candidate object, to obtain a second similarity matrix of Mx1 dimensions; performing clustering processing on the M candidate objects and the target object according to the first similarity matrix of MxM dimensions and the second similarity matrix of Mx1 dimensions, to obtain a plurality of clustering clusters; determining a clustering cluster in which the target object is located from the plurality of clustering clusters, to obtain a target cluster; determining the candidate object contained in the target cluster as the candidate object associated with the target object, to obtain at least one reference object.

3. The method of claim 2, wherein, The heart impact signal data of the target object comprises a plurality of groups of heart impact signal sub-data, and the M candidate objects collectively correspond to an emotion recognition sub-model, and each of the candidate objects has a corresponding emotion recognition sub-model; the determination of the second similarity between the target object and the candidate object using the heart impact signal data of the target object and the candidate object comprises: processing a plurality of groups of heart impact signal sub-data using the emotion recognition sub-models commonly corresponding to the M candidate objects to obtain first emotion category prediction values corresponding to each of the plurality of groups of heart impact signal sub-data, respectively; For the mth candidate object in the M candidate objects, an emotion recognition sub-model corresponding to the mth candidate object is used to process target ballistocardiogram sub-data to obtain a second emotion category prediction value, wherein the target ballistocardiogram sub-data is ballistocardiogram sub-data corresponding to target quantity of first emotion category prediction values in the plurality of first emotion category prediction values determined based on a preset ranking rule, and 1≤m≤M; According to the target first emotion category prediction value and the second emotion category prediction value, a second similarity between the target object and the mth candidate object is determined, wherein the target first emotion category prediction value is the maximum first emotion category prediction value in the plurality of first emotion category prediction values.

4. The method of claim 2, wherein, The determination of the first similarity between the two candidate objects by using the ballistocardiogram data and the emotion category label of each of the two candidate objects comprises: For the ath candidate object and the bth candidate object in the M candidate objects, an emotion recognition sub-model corresponding to the ath candidate object is used to process ballistocardiogram data of the bth candidate object to obtain a third emotion category prediction value of the bth candidate object, 1≤a≤M, 1≤b≤M; According to the third emotion category prediction value of the bth candidate object and the emotion category label of the bth candidate object, a first similarity between the ath candidate object and the bth candidate object is obtained.

5. The method of claim 1, wherein, The feature alignment processing of the ballistocardiogram data of the target object and the reference object based on the balanced distribution adaptive optimization function to obtain the converted signal data of the reference object comprises: According to the ballistocardiogram data of the target object, a target marginal probability distribution and a target conditional probability distribution are determined; According to the ballistocardiogram data of the reference object, a reference marginal probability distribution and a reference conditional probability distribution are determined; The target marginal probability distribution, the target conditional probability distribution, the reference marginal probability distribution and the reference conditional probability distribution are processed by using a balanced distribution adaptive optimization function to obtain a conversion matrix; According to the conversion matrix and the ballistocardiogram data of the reference object, the converted signal data of the reference object is obtained.

6. The method of claim 5, wherein, The processing of the target marginal probability distribution, the target conditional probability distribution, the reference marginal probability distribution and the reference conditional probability distribution by using a balanced distribution adaptive optimization function to obtain a conversion matrix comprises: The difference between the target marginal probability distribution and the reference marginal probability distribution is calculated by using a mean difference function to obtain a marginal difference value; The difference between the target conditional probability distribution and the reference conditional probability distribution is calculated by using the mean difference function to obtain a conditional difference value; The marginal difference value and the conditional difference value are processed by using a balanced distribution adaptive optimization function to obtain a conversion matrix, wherein the balanced distribution adaptive optimization function is used to optimize the distribution difference between the ballistocardiogram data of the target object and the reference object.

7. The method of claim 1, wherein, The emotion recognition model comprises N binary classifiers, N being a positive integer, the number of the binary classifiers being determined based on the number of sample emotion categories, and the emotion recognition model being trained based on the following operations: Obtaining training samples, the training samples comprising converted signal data of a sample reference object and a sample emotion category label of a sample target object, the sample reference object being an object associated with the sample target object; Performing feature extraction on the converted signal data of the sample reference object to obtain signal features; Using N binary classifiers to process the signal features respectively to obtain sample emotion category prediction values corresponding to each of the binary classifiers respectively; Performing statistics on the N sample emotion category prediction values based on a statistical function to obtain prediction scores corresponding to each sample emotion category; Determining an emotion category corresponding to a maximum prediction score among the prediction scores as a sample emotion recognition result, wherein the sample emotion recognition result represents an emotion category prediction result of the sample target object; Training an initial emotion recognition model based on the sample emotion recognition result and the sample emotion category label to obtain the emotion recognition model.

8. The method of claim 1, wherein, The ballistocardiogram data comprises at least one of the following: Heart rate variability data, respiratory variation data, beat-by-beat heartbeat statistical data, and beat-by-beat heartbeat nonlinear data.

9. An emotion recognition device based on ballistocardiogram, the device comprising: An obtaining module configured to obtain ballistocardiogram data of a target object and ballistocardiogram data of a plurality of candidate objects respectively; A screening module configured to perform similarity processing on the ballistocardiogram data of the target object and the ballistocardiogram data of the plurality of candidate objects respectively, to determine a candidate object associated with the target object from the plurality of candidate objects, and to obtain at least one reference object; An optimization module configured to perform feature alignment processing on the ballistocardiogram data of the target object and the reference object based on a balanced distribution adaptive optimization function, and to obtain converted signal data of the reference object; An identification module configured to process the ballistocardiogram data of the reference object using an emotion recognition model to obtain an emotion recognition result of the target object, and specifically configured to process the converted signal data of the reference object using the emotion recognition model to obtain the emotion recognition result of the target object.

Images