Emotion information identification method and device based on discrete labels, equipment and storage medium

Abstract

The invention discloses an emotional information recognition method, device and equipment based on discrete labels and a storage medium, relates to the technical field of emotional recognition, and discloses an emotional information recognition method based on discrete labels, and the method comprises the steps: carrying out the re-division of initial joint information according to a human body limb structure, and obtaining gait limb skeleton information; generating an initial discrete label according to the gait limb skeleton information; carrying out loss calculation on the gait limb skeleton information, the initial joint information and the initial discrete label, and optimizing the initial discrete label according to a loss calculation result to obtain a target discrete label; and performing multi-scale semantic alignment according to the target discrete label to obtain multi-scale emotion information so as to complete emotion information identification based on the discrete label. Based on the human body limb structure, joint information is re-divided, discrete labels and multi-scale semantic alignment are optimized, and the emotion information recognition accuracy and efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of emotion recognition technology, and in particular to methods, apparatus, devices and storage media for emotion information recognition based on discrete tags. Background Technology

[0002] Against the backdrop of rapid development in artificial intelligence and human-computer interaction technologies, emotion recognition has gradually become one of the important means to understand users' psychological states and behavioral patterns. The core of emotion recognition lies in extracting clues reflecting psychological changes from human behavioral data, thereby achieving intelligent judgment of emotion categories. Gait data, as an emerging non-contact biometric method, has advantages such as convenient collection, long recognition distance, and no need for active user cooperation, providing a new path for the large-scale deployment of emotion recognition systems in fields such as public safety, health monitoring, and emotion intervention.

[0003] Based on gait data for emotion recognition, researchers have attempted to improve the accuracy and stability of emotion recognition by using deep learning methods to model emotional features in gait. Early methods extracted emotional cues through manually designed motion features, and subsequently, graph neural networks and multi-view models were developed to enhance the modeling capabilities. However, traditional techniques typically rely on a large number of labeled gait samples, and the collection of emotion labels is not only costly but also highly subjective, making it difficult to meet the needs of large-scale training. To address the data labeling dependency problem, recent studies have introduced self-supervised learning frameworks to learn emotion representations from unlabeled samples. However, existing methods often neglect the high-level semantic structures contained in gait, resulting in significant shortcomings in modeling realistic and complex emotional expressions.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this application is to provide a method, apparatus, device, and storage medium for emotion information recognition based on discrete labels, aiming to solve the technical problem that existing gait emotion recognition methods cannot effectively model high-order emotion semantic information in self-supervised learning.

[0006] To achieve the above objectives, this application proposes a method for emotion information recognition based on discrete labels, the method comprising:

[0007] The initial joint information is re-divided based on the human limb structure to obtain gait limb skeleton information;

[0008] Initial discrete labels are generated based on the gait limb skeletal information;

[0009] The gait limb skeleton information, the initial joint information, and the initial discrete labels are used to calculate the loss, and the initial discrete labels are optimized based on the loss calculation results to obtain the target discrete labels;

[0010] Multi-scale semantic alignment is performed based on the target discrete labels to obtain multi-scale sentiment information, thereby completing sentiment information recognition based on discrete labels.

[0011] In one embodiment, the step of performing loss calculation on the gait limb skeleton information, the initial joint information, and the initial discrete labels, and optimizing the initial discrete labels based on the loss calculation results to obtain the target discrete labels includes:

[0012] The number of limb parts, the initial prediction result, and the uniformly distributed category data are obtained, wherein the initial prediction result is the result of the previous model prediction.

[0013] The gait limb skeletal information and the initial joint information are classified using a nearest neighbor algorithm to obtain a consistency score;

[0014] The supervision loss is calculated based on the number of limb parts, the consistency score, the initial discrete labels, the initial prediction results, and the uniformly distributed category data.

[0015] The process loss is calculated based on the consistency score, the initial prediction result, and the initial discrete label.

[0016] The classification loss is calculated based on the supervision loss and the process loss, and the initial discrete label is optimized based on the classification loss to obtain the target discrete label.

[0017] In one embodiment, the step of calculating the supervision loss based on the number of limb parts, the consistency score, the initial discrete labels, the initial prediction results, and the uniformly distributed category data includes:

[0018] Calculate the cross-entropy loss based on the initial discrete labels and the initial prediction results;

[0019] Calculate the divergence loss based on the uniformly distributed category data and the initial prediction results;

[0020] The supervision loss is calculated based on the number of limb parts, the cross-entropy loss, and the divergence loss.

[0021] In one embodiment, the step of generating initial discrete labels based on the gait limb skeletal information includes:

[0022] Cluster the gait limb skeleton information to generate the current hard discrete label;

[0023] Encoding calculations are performed based on the current hard discrete label to generate an initial discrete label.

[0024] In one embodiment, the step of performing multi-scale semantic alignment based on the discrete labels to obtain multi-scale sentiment information, thereby completing the sentiment information recognition based on discrete labels, includes:

[0025] Obtain skeleton nodes, body part nodes, and limb part nodes;

[0026] Based on the discrete labels, joint information, limb information, and motion information are obtained;

[0027] The mean square error is calculated based on the skeleton node, the body part node, the limb part node, the joint information, the limb information, and the motion information to obtain the mean square error calculation result for multiple scales. Then, the semantic alignment of multiple scales is performed based on the mean square error calculation result to obtain semantic information for multiple scales.

[0028] Calculate the inter-scale semantic loss based on the skeleton nodes, body part nodes, limb part nodes, joint information, limb information, and motion information;

[0029] The encoder parameters are updated based on the multi-scale semantic information and the inter-scale semantic loss, and multi-scale sentiment information is obtained based on the updated encoder parameters to complete the sentiment information recognition based on discrete labels.

[0030] In one embodiment, the step of calculating the inter-scale semantic loss based on the skeleton node, the body part node, the limb part node, the joint information, the limb information, and the motion information includes:

[0031] Calculate the average value of inter-scale information based on the joint information, the limb information, and the motion information;

[0032] The mean squared error is calculated based on the skeleton node, the body part node, the limb part node, and the average value of the inter-scale information to obtain the inter-scale mean squared error calculation result, and the inter-scale semantic loss is obtained based on the inter-scale mean squared error calculation result.

[0033] In one embodiment, the step of re-dividing the initial joint information based on the human limb structure to obtain gait limb skeletal information includes:

[0034] Based on the structure of the human limbs, the anatomical connection relationship based on the joints is obtained, and the initial joint information is divided into target structural regions according to the anatomical connection relationship. The target structural regions include the upper limb region, the lower limb region, and the trunk region.

[0035] The joint node sequence within the target structural region is reconstructed to construct a limb substructure diagram;

[0036] Temporal variation features are extracted from the limb substructure diagram to obtain gait limb skeleton information.

[0037] Furthermore, to achieve the above objectives, this application also proposes a discrete label-based emotion information recognition device, which includes:

[0038] The information segmentation module is used to re-segment the initial joint information according to the human limb structure to obtain gait limb skeleton information;

[0039] The tag generation module is used to generate initial discrete tags based on the gait limb skeleton information;

[0040] The loss calculation module is used to perform loss calculation on the gait limb skeleton information, the initial joint information and the discrete labels, and optimize the initial discrete labels according to the loss calculation results to obtain the target discrete labels;

[0041] The information recognition module is used to perform multi-scale semantic alignment based on the target discrete label to obtain multi-scale sentiment information, so as to complete the sentiment information recognition based on discrete label.

[0042] Furthermore, to achieve the above objectives, this application also proposes a discrete tag-based emotion information recognition device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the discrete tag-based emotion information recognition method described above.

[0043] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the emotion information recognition method based on discrete tags as described above.

[0044] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the discrete label-based emotion information recognition method described above.

[0045] One or more technical solutions proposed in this application have at least the following technical effects:

[0046] This technique addresses several issues related to emotional information recognition. First, it re-divides initial joint information based on human limb structure to obtain gait skeletal information. Second, it generates initial discrete labels based on the gait skeletal limb and joint information. Third, it calculates the loss of the initial joint information and initial discrete labels, optimizes the initial discrete labels based on the loss calculation results to obtain target discrete labels, and finally performs multi-scale semantic alignment based on the target discrete labels to obtain multi-scale emotional information. This solves problems such as the initial joint information failing to reflect the inter-limb relationships, resulting in inaccurate gait representation; insufficient accuracy of discrete labels preventing precise matching of gait features and emotional information; and low emotional information recognition accuracy failing to fully mine and integrate multi-scale emotional features. Compared to existing technologies, this technique improves the accuracy, robustness, and efficiency of emotional information recognition. The optimized discrete labels are closer to reality and better reflect the corresponding relationships. Multi-scale semantic alignment fully mines and integrates emotional features, making the recognition results more accurate and reliable. It also exhibits stronger adaptability to gait changes, can accurately recognize even in the face of noise interference, reduces computational redundancy, accelerates recognition speed, and meets the requirements for real-time high efficiency. Attached Figure Description

[0047] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart illustrating an embodiment of the emotion information recognition method based on discrete labels provided in this application.

[0050] Figure 2 This is a flowchart illustrating Embodiment 2 of the emotion information recognition method based on discrete labels provided in this application;

[0051] Figure 3 A simplified flowchart illustrating the emotion information recognition method based on discrete labels provided in Embodiment 2 of this application;

[0052] Figure 4 This is a schematic diagram of the module structure of the emotion information recognition device based on discrete tags according to an embodiment of this application;

[0053] Figure 5This is a schematic diagram of the device structure of the hardware operating environment involved in the emotion information recognition method based on discrete tags in the embodiments of this application.

[0054] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0055] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0056] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0057] The main solution of this application embodiment is as follows: re-divide the initial joint information according to the human limb structure to obtain gait limb skeleton information; generate initial discrete labels according to the gait limb skeleton information; perform loss calculation on the gait limb skeleton information, the initial joint information and the initial discrete labels, and optimize the initial discrete labels according to the loss calculation results to obtain target discrete labels; perform multi-scale semantic alignment according to the target discrete labels to obtain multi-scale sentiment information, so as to complete the sentiment information recognition based on discrete labels.

[0058] In this embodiment, for ease of description, the following description will focus on an emotion recognition device based on discrete tags.

[0059] Because existing gait emotion recognition methods cannot effectively model high-order emotional semantic information in self-supervised learning, this application provides a solution. This solution employs a technique that re-divides initial joint information based on human limb structure to obtain gait limb skeleton information; generates initial discrete labels based on the limb and joint information contained in the gait skeleton; calculates loss using the initial joint information and initial discrete labels; optimizes the initial discrete labels based on the loss calculation results to obtain target discrete labels; and then performs multi-scale semantic alignment based on the target discrete labels to obtain multi-scale emotional information, thus achieving emotional information recognition based on discrete labels. This addresses the issue that initial joint information cannot reflect the relationships between limbs. The existing technologies address several issues, including the inability to accurately reflect human gait due to associated information, insufficient accuracy of discrete labels leading to inaccurate matching of gait features and emotional information, and low accuracy in emotional information recognition failing to fully mine and integrate multi-scale emotional features. Compared with existing technologies, this approach improves the accuracy, robustness, and efficiency of emotional information recognition. The optimized discrete labels are closer to reality and better reflect the corresponding relationships. Multi-scale semantic alignment can fully mine and integrate emotional features, making the recognition results more accurate and reliable. It is also more adaptable to gait changes, can be relatively accurate in the face of noise interference, reduces computational redundancy, speeds up the recognition process, and meets the requirements for real-time high efficiency.

[0060] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, such as a discrete label-based emotion information recognition device. The following description uses a discrete label-based emotion information recognition device as an example to illustrate this embodiment and the subsequent embodiments.

[0061] Based on this, embodiments of this application provide a method for emotion information recognition based on discrete tags, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the emotion information recognition method based on discrete labels in this application.

[0062] In this embodiment, the emotion information recognition method based on discrete labels includes steps S10 to S40:

[0063] Step S10: The initial joint information is re-divided according to the human limb structure to obtain gait limb skeleton information;

[0064] It should be noted that the human limb structure refers to the different limb regions divided according to the physiological structure of the human body. For example, it can be divided into the core body part, such as the trunk, and the limb parts, including the upper limbs and lower limbs. In this embodiment, it will be specifically divided into "body parts" and "limb parts" to distinguish the different roles of different regions in the expression of emotions. For example, the limbs often have more obvious movement changes than the trunk when expressing emotions.

[0065] Additionally, it should be noted that initial joint information refers to the basic data contained in the raw gait data, including the position and movement trajectory of each joint in the human body. This data is typically presented as a three-dimensional time series, covering the dynamic changes of all joints during walking. This information comes directly from the gait acquisition device and serves as the raw input for emotion recognition, containing fundamental details of human movement.

[0066] Furthermore, gait skeletal information is structured data obtained by re-dividing the initial joint information according to the human limb structure. In this embodiment, it is represented by body part data and limb part data, where the body part data contains a certain number of key joints, and the limb part data also contains a corresponding number of key joints. It not only preserves the dynamic characteristics of the original joints, but also reflects the structural relationships between different limb regions, and can more clearly reflect the coordinated movement characteristics of the limbs in emotional expression.

[0067] Understandably, the initial joint information is acquired, including data on the position of all joints in three-dimensional space over time during human walking. Based on the physiological differences in human limb structure, this initial joint information is divided into regions. Specifically, key joints related to the trunk, such as the hip and lumbar joints, are classified as body parts, while joints related to upper and lower limb movement, such as the shoulder, elbow, knee, and ankle joints, are classified as limb parts. Through this division, the originally scattered joint information is integrated into limb region data with clear physiological significance, resulting in gait skeletal information.

[0068] Step S20: Generate initial discrete labels based on the gait limb skeleton information;

[0069] It should be noted that gait skeletal information is structured joint data after being divided according to the human limb structure. It includes the dynamic characteristics of body parts and limb parts, which not only preserves the original joint position and movement information, but also incorporates the structural relationships of limb regions.

[0070] Furthermore, the initial discrete labels are preliminary labels obtained by classifying unlabeled gait data based on gait skeletal information using a clustering algorithm. These labels exist in discrete form and are used to represent the initial classification of data samples in terms of emotional expression. They serve as initial pseudo-labels to guide model training in self-supervised learning.

[0071] Understandably, gait skeletal information is used as input, and cluster analysis is performed on the unlabeled samples. The core of clustering is to group samples with similar emotional expression features into one category based on the similarity of different samples in gait skeletal information. In this embodiment, a density clustering algorithm is used to achieve this process. This algorithm automatically identifies sample clusters with similar features in gait skeletal information by setting an appropriate density threshold. Each cluster represents a potential emotional category, and then a corresponding discrete value is assigned to each sample as an initial label.

[0072] In one feasible implementation, step S20 may include steps S21 to S22:

[0073] Step S21: Cluster the gait limb skeleton information to generate the current hard discrete label;

[0074] It should be noted that clustering is an unsupervised learning method used to automatically group data samples with similar characteristics into one category. In this embodiment, the object of clustering is gait skeletal information (joint dynamic data divided according to human limb structure). The core is to aggregate samples with similar emotional expressions into multiple groups based on the similarity of different samples in gait features, such as limb movement amplitude, rhythm, trajectory, etc.

[0075] Furthermore, the current hard discrete label refers to the preliminary label obtained through clustering. "Hard" means that each sample is explicitly assigned to a unique category, rather than belonging to multiple categories in a probabilistic form. The label exists in the form of discrete numerical values ​​or symbols, such as 0, 1, 2, etc., and each numerical value corresponds to a potential sentiment category.

[0076] Understandably, gait skeletal information is used as input, and clustering algorithms (such as DBSCAN density clustering) are employed to group unlabeled samples. During clustering, the algorithm automatically identifies samples with similar features in the gait skeletal information: for example, a "happy" gait is typically characterized by large limb movements and a brisk pace, while a "sad" gait is characterized by small limb movements and a slow pace. The algorithm will cluster samples with similar features into one category. After clustering, each category is assigned a unique discrete value as a label, i.e., the current hard discrete label. For example, the sample group related to "happy" is labeled as 0, the sample group related to "angry" is labeled as 1, and the sample group related to "sad" is labeled as 2. Each sample will be assigned a corresponding label, forming the current hard discrete label.

[0077] Step S22: Perform encoding calculations based on the current hard discrete label to generate an initial discrete label.

[0078] It should be noted that encoding computation refers to the process of processing the sample features corresponding to the current hard discrete label through an encoder, extracting more discriminative emotional features, and further optimizing the label based on these features. The core of encoding computation is to enable the encoder to learn the mapping relationship between the current hard discrete label and gait skeletal information, thereby enhancing the representativeness of the label in terms of emotional features.

[0079] Understandably, the current hard discrete labels are used as supervisory signals and input into the encoder for training. The encoder learns the emotional feature patterns corresponding to each label based on gait skeletal information. Through encoding calculations, the encoder re-predicts the emotional category of the samples, correcting the bias caused by clustering noise in the current hard discrete labels. The final prediction result is the initial discrete label, which further improves the matching degree between the label and the emotional features based on the current hard discrete labels.

[0080] Step S30: Perform loss calculation on the gait limb skeleton information, the initial joint information and the initial discrete label, and optimize the initial discrete label according to the loss calculation result to obtain the target discrete label;

[0081] It should be noted that loss calculation refers to measuring the degree of difference between gait limb skeletal information, initial joint information, and initial discrete labels through a preset calculation method, thereby evaluating the accuracy of the initial discrete labels. In this embodiment, loss calculation combines cross-entropy and KL divergence, and introduces consistency score as a weight to form a comprehensive loss assessment.

[0082] Furthermore, the target discrete label is a more accurate label obtained by iteratively optimizing the initial discrete label using the loss calculation results. It corrects the bias caused by clustering noise or redundant information in the initial label, more closely reflects the true distribution of sentiment categories, and has stronger discriminative power.

[0083] Furthermore, the consistency score measures the similarity between the initial joint information and the body part features in the gait skeletal information. It is obtained by calculating the ratio of the intersection to the union of the Top-k similar sample sets of the initial joint information and the Top-k similar sample sets of the body parts, with a value between 0 and 1. The closer the value is to 1, the higher the consistency between the two in terms of emotional features.

[0084] Understandably, a consistency score is calculated. KNN classification is applied to body parts in the initial joint information and gait skeletal information, respectively. Top-k similar samples are selected for each, and the consistency score is obtained by the ratio of the intersection to the union of these two sample sets, reflecting their consistency in emotional features. Loss is calculated based on the consistency score. On one hand, cross-entropy is used to measure the difference between the initial discrete labels and the predicted body parts, and the consistency score is used as the weight for this part. On the other hand, KL divergence is used to measure the difference between the predicted body parts and the uniform distribution, and 1 minus the consistency score is used as the weight for this part, thus forming the body part loss. Simultaneously, by aggregating the prediction results of limb features, the consistency score is used as a weight to fuse the prediction results of body parts and initial discrete labels, calculating the aggregate loss. The final comprehensive loss is the sum of the body part loss and the aggregate loss.

[0085] In one feasible implementation, step S30 may include steps S31 to S35:

[0086] Step S31: Obtain the number of limb parts, the initial prediction result, and the uniformly distributed category data, wherein the initial prediction result is the result of the previous model prediction;

[0087] It should be noted that the number of limb parts refers to the number of key joints or regions contained in a limb part. For example, after dividing the human limb structure, the total number of joints contained in a limb part is used to quantify the scale of limb features and serves as a reference for sample size in loss calculation.

[0088] Additionally, the initial prediction result refers to the prediction result made by the model for the emotion category corresponding to the gait data in the previous training iteration, such as predicting a gait sample as "angry" or "happy", which exists in the form of discrete labels, such as values ​​0, 1, etc.

[0089] Furthermore, uniformly distributed categorical data refers to an assumed distribution of sentiment categories where each sentiment category has an equal probability. For example, if there are three sentiment categories, each category has a probability of 1 / 3. This serves as a baseline distribution to measure whether the model's predictions are overly biased towards a particular category, thus avoiding biased predictions due to data imbalance.

[0090] Understandably, determining the number of limb parts involves counting the limb parts identified in step S10, such as the specific number of key joints or regions contained in the upper and lower limbs. For example, if a limb part contains 10 joints, then the number of limb parts is 10. Extracting the initial prediction results involves calling the sentiment category prediction labels for all samples from the previous iteration of the model; these labels are generated by the model based on previous feature learning. Generating uniformly distributed category data involves setting the probability of each category to be equal, such as 1 / 3, based on the actual total number of sentiment categories (e.g., 3 categories), as a benchmark for subsequently judging whether the prediction results are balanced.

[0091] Step S32: Classify the gait limb skeleton information and the initial joint information using a nearest neighbor algorithm to obtain a consistency score;

[0092] It should be noted that nearest neighbor classification refers to using the K-Nearest Neighbors (KNN) algorithm to find the K most similar samples for each sample, i.e., the "nearest neighbors," and to determine the class tendency of the sample based on the class distribution of the nearest neighbors. For example, for a certain sample, if the five most similar samples are found in the gait skeletal information, and four of them belong to "happy," then the sample tends to be "happy."

[0093] Furthermore, the consistency score refers to the degree of overlap between the gait skeletal information and the initial joint information in the set of similar samples obtained by the nearest neighbor algorithm classification, and is used to measure the consistency of the two in terms of emotional features. The closer the value is to 1, the more similar the emotional features captured by the two are; the closer it is to 0, the greater the difference.

[0094] It is understandable that the consistency score is calculated using the following formula:

[0095]

[0096] In the formula, X∈R 3×T×J X is a component node in the skeleton data. HA ∈R 3×T×5 These are nodes representing body parts. Additionally, X... LA ∈R 3×T×10 These are nodes representing limb parts.

[0097] Step S33: Calculate the supervision loss based on the number of limb parts, the consistency score, the initial discrete labels, the initial prediction results, and the uniformly distributed category data;

[0098] It should be noted that supervised loss is a quantitative metric used to evaluate the difference between the initial discrete labels and the model's prediction results. It aims to judge the accuracy of the initial labels by the magnitude of the loss value and guide the labels to be optimized in a direction that is more in line with real emotions.

[0099] Understandably, the supervised loss is calculated using the number of limb parts as the sample size benchmark, combined with the consistency score, initial discrete labels, initial prediction results, and uniformly distributed class data. When the consistency score is high, such as 0.8, it indicates that the limb part features are consistent with the original features. In this case, cross-entropy, which measures the difference between the initial discrete labels and the initial prediction results, is used as the main loss term, with a weight equal to the consistency score. When the consistency score is low, such as 0.3, it indicates that the limb part features may be affected by redundant information. In this case, KL divergence, which measures the difference between the initial prediction results and the uniform distribution, is used as a supplementary loss term, with a weight of 1 minus the consistency score. The supervised loss is obtained by averaging the two losses over the number of limb parts.

[0100] In one feasible implementation, step S33 may include: calculating cross-entropy loss based on the initial discrete labels and the initial prediction results; calculating divergence loss based on the uniformly distributed category data and the initial prediction results; and calculating supervision loss based on the number of limb parts, the cross-entropy loss, and the divergence loss.

[0101] It should be noted that cross-entropy loss is a quantitative metric used to measure the difference between two probability distributions. In this embodiment, it is used to evaluate the difference between the initial discrete labels, representing the expected sentiment category of the sample, and the initial prediction result, i.e., the model's prediction of the sample's sentiment category. The greater the difference, the higher the cross-entropy loss value; the smaller the difference, the lower the loss value.

[0102] Additionally, in this embodiment, divergence loss refers to KL divergence loss, which measures the difference between the model's predicted distribution and the baseline distribution of uniformly distributed class data, where the probabilities of each sentiment class are equal.

[0103] Understandably, the cross-entropy loss and KL divergence are calculated using the following supervised loss formula:

[0104]

[0105] In the formula, N HA Represents the number of body parts, u represents a uniform vector, and y represents the number of body parts. HA To predict the outcome, Y s For discrete labels, CE and KL represent cross-entropy and KL divergence loss, respectively, and u represents uniform distribution.

[0106] Step S34: Calculate the process loss based on the consistency score, the initial prediction result, and the initial discrete label;

[0107] It should be noted that process loss refers to a quantitative indicator that evaluates the stability of labels during the iterative process by fusing the prediction results of limb features with the initial discrete labels. It aims to ensure that the prediction results from different feature sources remain consistent during the label optimization process.

[0108] Understandably, the consistency score is used as the weight to fuse the initial prediction results, and the process loss is calculated based on the limb part features and the initial discrete labels. A high consistency score indicates reliable limb part predictions and is assigned a higher weight, such as 0.8; a low consistency score indicates that the initial discrete labels may be closer to the original features and is assigned a higher weight, such as 0.7. Finally, the difference between the fused prediction results and the initial discrete labels is calculated using cross-entropy to obtain the process loss. The process loss is calculated as follows:

[0109]

[0110] In the formula, N represents the number of data points in the dataset; S represents the consistency score; Y s For discrete labels; y HA For the predicted result; y s This represents a refined soft label, used to mitigate noise from the initial cluster labels and improve sensitivity to key emotional semantic regions (such as limbs).

[0111] Step S35: Calculate the classification loss based on the supervision loss and the process loss, and optimize the initial discrete label based on the classification loss to obtain the target discrete label.

[0112] It should be noted that classification loss refers to the sum of supervision loss and process loss. It is a comprehensive indicator that measures the overall accuracy of the initial discrete labels. The smaller the loss value, the closer the labels are to the true sentiment category.

[0113] Understandably, the classification loss is obtained by adding the supervision loss and the process loss. A large classification loss indicates a significant deviation between the initial labels and the model's predictions or feature distributions, requiring adjustment of the sample's class assignment. A small classification loss indicates relatively accurate labels, and the current labels should be retained. The classification loss is calculated using the following formula:

[0114] L class =L HA +l

[0115] In the formula, l represents process loss; L HA This indicates a loss of oversight.

[0116] Step S40: Perform multi-scale semantic alignment based on the target discrete label to obtain multi-scale sentiment information, thereby completing the sentiment information recognition based on discrete labels.

[0117] It should be noted that multi-scale semantic alignment refers to ensuring semantic consistency of features at different scales, such as joints, bones, and motion, through a preset loss calculation method, while clearly defining the boundaries of different emotion categories. In this embodiment, it includes intra-scale semantic alignment and inter-scale semantic alignment. Intra-scale semantic alignment ensures consistency between single features and multiple features at the same scale, while inter-scale semantic alignment is used to separate the feature distributions of different emotion categories.

[0118] Furthermore, multi-scale emotional information refers to emotional representations that integrate features at different scales, such as joints, bones, and movement. It includes both the dynamic details of local joints and the overall movement trends of the limbs, and can comprehensively reflect the emotional cues contained in the gait.

[0119] Additionally, emotion recognition refers to determining the emotion category corresponding to gait data, such as joy, anger, or sadness, based on multi-scale emotion information. It is the final classification task completed using discrete labels of the target and multi-scale features.

[0120] Understandably, calculating the mean squared error of single-scale features and multi-scale features ensures semantic consistency of the same sentiment category under different feature combinations. Calculating the mean squared error of a single-scale feature and the average of all scale features clarifies category boundaries by widening the feature distance between different sentiment categories. Through this alignment process, features from different scales, such as joints, skeletons, and motion, are fused to form multi-scale sentiment information. Finally, based on the target discrete label and multi-scale sentiment information, a classifier determines the sentiment category corresponding to each sample, completing the sentiment information recognition.

[0121] In one feasible implementation, step S40 may include steps S41 to S45:

[0122] Step S41: Obtain skeleton nodes, body part nodes, and limb part nodes;

[0123] It should be noted that skeletal nodes refer to all the joints that make up the complete human gait skeleton, including all the key joints involved in movement during walking, such as the head, neck, shoulders, elbows, wrists, hips, knees, and ankles. The trajectory of the position changes of these nodes over time forms the basis of the original gait data and can reflect the overall movement state of the human body.

[0124] Additionally, body part nodes refer to the core areas of the body after dividing it according to the human limb structure, such as the joint nodes of the trunk and adjacent parts, including the thoracic vertebrae, lumbar vertebrae, pelvis, and hip joints. These nodes usually have a small range of motion in emotional expression and mainly play a supporting role, but their positional changes can still provide auxiliary information for emotion recognition.

[0125] Additionally, limb joint nodes refer to the joints belonging to the limbs after dividing the human body into structural regions, including the shoulder, elbow, and wrist joints of the upper limbs, and the hip, knee, and ankle joints of the lower limbs. These joints have a large range of motion in emotional expression and are key to conveying emotional characteristics.

[0126] Step S42: Based on the discrete labels, obtain joint information, limb information, and motion information;

[0127] It should be noted that joint information refers to the dynamic characteristic data of a single joint point, including the joint's position coordinates in three-dimensional space, the rate of change of position over time, and the range of motion, such as the changes in the X, Y, and Z coordinates and the swing amplitude of the elbow joint during walking.

[0128] Furthermore, limb information refers to the overall characteristic data of a limb part composed of multiple adjacent joints, including changes in limb length, overall movement trajectory, and coordinated movement patterns of various joints in the limb, such as the overall swing trajectory and movement coordination of the entire upper limb.

[0129] In addition, motion information refers to dynamic motion trend data of joints and limbs, including motion rhythm, such as the number of swings per second, motion acceleration, and changes in motion direction. For example, in a "happy" gait, the limbs move with a brisk rhythm, while in a "sad" gait, the movement rhythm is slow.

[0130] Understandably, this involves extracting emotion-related features from gait data using discrete target labels as a reference. This includes extracting data such as position, velocity, and amplitude at each joint to form joint information, focusing on the subtle movement features of individual joints; integrating features from multiple joints within the same limb to calculate the overall movement trajectory and coordinated movement patterns of the limb, forming limb information that reflects the overall movement state of the limb; and analyzing the temporal characteristics of joint and limb movements, such as rhythm, acceleration, and directional changes, to form motion information and capture the dynamic trends of movement.

[0131] Step S43: Calculate the mean square error based on the skeleton node, the body part node, the limb part node, the joint information, the limb information and the motion information to obtain the mean square error calculation result for multiple scales, and perform semantic alignment for multiple scales based on the mean square error calculation result for multiple scales to obtain semantic information for multiple scales.

[0132] It should be noted that mean squared error (MSE) is an indicator that measures the difference between two data sequences. It is calculated as the average of the squares of the differences between corresponding elements in the two sequences. In this embodiment, it is used to measure the consistency of different feature combinations at the same scale. The smaller the difference, the lower the MSE value, indicating that the feature semantics are more consistent.

[0133] Understandably, for different scales, such as joint scale, limb scale, and motion scale, the mean square error of a single feature and the combination of multiple features are calculated separately. After obtaining the mean square error calculation results for multiple scales, the error is reduced by adjusting the feature weights, so that the sentiment semantics expressed by the single feature and the combined feature at the same scale are consistent, thus completing the semantic alignment of multiple scales and ultimately forming semantic information of multiple scales. The calculation formula for semantic alignment of multiple scales is as follows:

[0134] L intra =MSE(y single ,y multi ),y∈(y J ,y B ,y M )

[0135] In the formula, MSE(·) represents the mean square error, and y single ,y multi Corresponding to X respectively single ,X mulyi X single Let X represent X, X LA and X HA Corresponding joint (J) or bone (B) or motion (M) information; X multi Let X represent X, X LA and X HA The corresponding joint (J), bone (B), and motion (M) information.

[0136] Step S44: Calculate the inter-scale semantic loss based on the skeleton node, the body part node, the limb part node, the joint information, the limb information, and the motion information;

[0137] It should be noted that inter-scale refers to the cross-scale relationship between joint scale, limb scale, and motion scale, such as the semantic differences between joint information and limb information, and limb information and motion information. Furthermore, inter-scale semantic loss refers to an indicator used to measure the differences in emotional semantics between features at different scales. The purpose is to widen the feature distance between different emotion categories, clarify category boundaries, and make the feature distribution of categories such as "anger," "happiness," and "sadness" easier to distinguish.

[0138] Understandably, the average feature values ​​at the joint, limb, and motion scales are calculated and used as the baseline features for that scale. The mean squared error of a single feature at each scale compared to the average of the baseline features at the other two scales is then calculated and used as the inter-scale semantic loss.

[0139] In one feasible implementation, step S44 may include: calculating the average value of inter-scale information based on the joint information, the limb information, and the motion information; calculating the mean square error based on the skeleton node, the body part node, the limb part node, and the average value of inter-scale information to obtain the inter-scale mean square error calculation result; and obtaining the inter-scale semantic loss based on the inter-scale mean square error calculation result.

[0140] It should be noted that the average value of information across scales refers to the comprehensive value obtained by averaging the feature data of three different scales: joint information, limb information, and motion information. It is used as a benchmark to measure the differences in features at different scales, so that features at each scale can be compared under the same benchmark.

[0141] Understandably, the calculation of inter-scale semantic loss is as follows:

[0142] L inter =MSE(y single ,η),y∈(y J ,y B ,y M )

[0143] In the formula, η represents y J ,y B ,y M The average value of y single Corresponding to X single X single Let X represent X, X LA and X HA The corresponding joint (J), bone (B), or motion (M) information.

[0144] Step S45: Update the encoder parameters based on the multi-scale semantic information and the inter-scale semantic loss, and obtain multi-scale sentiment information based on the updated encoder parameters to complete the sentiment information recognition based on discrete labels.

[0145] It should be noted that encoder parameters refer to the encoder used to extract and process gait features, such as weights and biases in a neural network model. These parameters determine the encoder's ability to extract features; the better the parameters, the more the extracted features reflect the essence of emotion.

[0146] Understandably, multi-scale semantic information and inter-scale semantic loss are input into the encoder, and the encoder parameters are adjusted through backpropagation. If the mean squared error of the multi-scale semantic information is high, it indicates inconsistency in features at the same scale, requiring parameter adjustment to enhance feature consistency. If the inter-scale semantic loss is high, it indicates blurred category boundaries, requiring parameter adjustment to widen category differences. After parameter updates, the encoder can more accurately extract multi-scale sentiment information that integrates joint, limb, and motion information. Finally, based on the multi-scale sentiment information and discrete target labels, a classifier, such as a fully connected layer, determines the sentiment category of each sample, completing the recognition process.

[0147] This embodiment provides a method for emotion information recognition based on discrete labels. It obtains gait limb skeleton information by re-dividing initial joint information according to human limb structure, generates initial discrete labels, calculates loss and optimizes the initial discrete labels to obtain target discrete labels, and performs multi-scale semantic alignment to obtain multi-scale emotion information. This method solves the problems of low accuracy of emotion recognition based on gait data, insufficient label optimization, and inadequate feature fusion in the prior art, and achieves the beneficial effects of improving the accuracy of emotion recognition, enhancing the label optimization effect, and improving the quality of feature fusion.

[0148] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Step S10 of the discrete label-based emotion information recognition method includes steps S11 to S13:

[0149] Step S11: Based on the human limb structure, obtain the anatomical connection relationship based on the joints, and divide the initial joint information into target structural regions according to the anatomical connection relationship. The target structural regions include the upper limb region, the lower limb region, and the trunk region.

[0150] It should be noted that the anatomical connections between joints refer to the inherent physiological connections between human joints, that is, the natural connections formed between adjacent joints through bones, muscles, or ligaments. For example, the shoulder joint connects the trunk and upper limb, the hip joint connects the trunk and lower limb, and the elbow joint connects the upper arm and forearm. This connection determines the synergy and limitations of joint movement and is the basis of limb movement characteristics.

[0151] Furthermore, the target structural region refers to the limb region with clear function and structural boundaries, divided according to anatomical connections. This includes: the upper limb region, which consists of joints related to arm movement, such as the shoulder joint, elbow joint, and wrist joint; the lower limb region, which consists of joints related to leg movement, such as the hip joint, knee joint, and ankle joint; and the trunk region, which consists of joints that constitute the core of the body, such as the thoracic vertebrae, lumbar vertebrae, and pelvis.

[0152] Understandably, based on the physiological characteristics of the human limb structure, the anatomical connections between joints are clarified. For example, the inherent connections between the shoulder joint and the trunk, and between the elbow joint and the shoulder and wrist joints, are identified. Based on these connections, all joints in the initial joint information are classified: joints related to arm movement are classified into the upper limb region; joints related to leg movement are classified into the lower limb region; and joints constituting the core of the body are classified into the trunk region.

[0153] Step S12: Reconstruct the joint node sequence within the target structural region to build a limb substructure diagram;

[0154] It is understandable that the sequence of joint nodes within the target structural region refers to the set of joints contained in each target structural region arranged in anatomical connection order.

[0155] Furthermore, structural reconstruction refers to re-organizing and strengthening the connections between joints within a region based on the anatomical connections of the joint node sequence, clarifying the hierarchical relationships of joints and the motion transmission pathways. For example, in the lower limb region, it strengthens the motion transmission connections between the hip joint and the knee joint, and between the knee joint and the ankle joint.

[0156] Furthermore, a limb substructure diagram refers to a visualized or data-driven joint connection diagram formed after structural reconstruction of each target structural region. The diagram clearly shows the connection methods and motion dependencies of the joints within the region. For example, the movement of the elbow joint depends on the movement of the shoulder joint. It is a structured expression of the coordinated movement characteristics of the joints within the region.

[0157] Furthermore, it is understandable that for each target structural region, the sequence of joint nodes within it is first obtained, and then the structure is reconstructed based on the motion dependencies of joints in anatomy, constructing a limb substructure diagram for each region. Taking the upper limb region as an example, the substructure diagram will clearly show the connection lines and motion transmission directions between the shoulder and elbow joints, and between the elbow and wrist joints.

[0158] Step S13: Extract the temporal change features from the limb substructure diagram to obtain gait limb skeleton information.

[0159] Understandably, for each limb substructure diagram, such as the upper limb, lower limb, and trunk substructure diagrams, the temporal change features of the joints are extracted. Then, the temporal change features of the upper limb, lower limb, and trunk substructure diagrams are integrated to form structured data that includes the dynamic features of all regions of the body and the collaborative relationships between regions, namely, gait limb skeleton information.

[0160] This embodiment provides a discrete label-based emotion information recognition method. By obtaining the anatomical connection relationship based on the joints according to the human limb structure, dividing the initial joint information into target structural regions, reconstructing the joint node sequence within the target structural regions to construct a limb substructure map, and extracting the temporal change features from the limb substructure map to obtain gait limb skeleton information, this method solves the technical problems of insufficient joint information division, insufficient expression of limb movement features, and insufficient gait data structuring in traditional methods. It achieves the beneficial effects of improving the accuracy of emotion recognition based on gait data, enhancing the expression of limb movement features, and improving the structuring quality of gait data.

[0161] For example, to help understand the implementation process of the discrete label-based emotion information recognition method obtained by combining this embodiment with the above embodiment one, please refer to... Figure 3 , Figure 3 A simplified flowchart of a sentiment information recognition method based on discrete labels is provided, specifically:

[0162] The top left corner of the figure shows the original skeleton extracted from joint, bone, or motion information, which can be J(J), B(B), or M(M) skeletons. Next, an algorithm called Apperance-Skeleton (ASR) is used to rearrange and re-arrange the skeleton nodes, generating low-high (LA) and high-high (HA) level skeleton information. This skeleton information is then fed into an encoder, which processes the information using shared weights. At the encoder's output, we obtain J(J) and B(B) skeleton features, as well as J&BAM-SkeletonsFeatures. These features are then fed into a module called Multi-Granularity Appending Alignment (MGAA), where they are processed using an LSTM (Long Short-Term Memory) network. Next, on the right side, we see Apperance-refined Multi-scale Discrete Label (AMDL), which is an optimized multi-scale discrete label. It includes Apperance-Skeleton Partitioning (ASP) features and Global-Context Features. These features are processed through a fully connected layer (FC) to generate Hard Discrete Labels and Soft Discrete Labels. Finally, these labels are further optimized using a module called Apperance Label Refinement (ALR) to improve the accuracy of sentiment recognition.

[0163] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the emotional information recognition method based on discrete labels in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0164] This application also provides a sentiment information recognition device based on discrete tags, please refer to... Figure 4 The emotion information recognition device based on discrete tags includes:

[0165] The information segmentation module 10 is used to re-segment the initial joint information according to the human limb structure to obtain gait limb skeleton information;

[0166] The tag generation module 20 is used to generate initial discrete tags based on the gait limb skeleton information;

[0167] The loss calculation module 30 is used to perform loss calculation on the gait limb skeleton information, the initial joint information and the discrete label, and optimize the initial discrete label according to the loss calculation result to obtain the target discrete label;

[0168] The information recognition module 40 is used to perform multi-scale semantic alignment based on the target discrete label to obtain multi-scale sentiment information, so as to complete the sentiment information recognition based on discrete label.

[0169] The discrete label-based emotion recognition device provided in this application, employing the discrete label-based emotion recognition method described in the above embodiments, can solve the technical problem that existing gait emotion recognition methods cannot effectively model high-order emotion semantic information in self-supervised learning. Compared with the prior art, the beneficial effects of the discrete label-based emotion recognition device provided in this application are the same as those of the discrete label-based emotion recognition method provided in the above embodiments, and other technical features in the discrete label-based emotion recognition device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0170] In one embodiment, the loss calculation module 30 is further configured to acquire the number of limb parts, the initial prediction result, and uniformly distributed category data, wherein the initial prediction result is the result of the previous model prediction; classify the gait limb skeleton information and the initial joint information using a nearest neighbor algorithm to obtain a consistency score; calculate a supervised loss based on the number of limb parts, the consistency score, the initial discrete label, the initial prediction result, and the uniformly distributed category data; calculate a process loss based on the consistency score, the initial prediction result, and the initial discrete label; calculate a classification loss based on the supervised loss and the process loss, and optimize the initial discrete label based on the classification loss to obtain a target discrete label.

[0171] In one embodiment, the loss calculation module 30 is further configured to calculate cross-entropy loss based on the initial discrete labels and the initial prediction results; calculate divergence loss based on the uniformly distributed category data and the initial prediction results; and calculate supervision loss based on the number of limb parts, the cross-entropy loss, and the divergence loss.

[0172] In one embodiment, the label generation module 20 is further configured to cluster the gait limb skeleton information to generate a current hard discrete label; and to perform encoding calculations based on the current hard discrete label to generate an initial discrete label.

[0173] In one embodiment, the information recognition module 40 is further configured to acquire skeleton nodes, body part nodes, and limb part nodes; obtain joint information, limb information, and motion information based on the discrete labels; calculate the mean square error based on the skeleton nodes, body part nodes, limb part nodes, joint information, limb information, and motion information to obtain a multi-scale mean square error calculation result; perform multi-scale semantic alignment based on the multi-scale mean square error calculation result to obtain multi-scale semantic information; calculate the inter-scale semantic loss based on the skeleton nodes, body part nodes, limb part nodes, joint information, limb information, and motion information; update encoder parameters based on the multi-scale semantic information and the inter-scale semantic loss; and obtain multi-scale sentiment information based on the updated encoder parameters to complete sentiment information recognition based on discrete labels.

[0174] In one embodiment, the information recognition module 40 is further configured to calculate the average value of inter-scale information based on the joint information, the limb information, and the motion information; calculate the mean square error based on the skeleton node, the body part node, the limb part node, and the average value of inter-scale information to obtain the inter-scale mean square error calculation result; and obtain the inter-scale semantic loss based on the inter-scale mean square error calculation result.

[0175] In one embodiment, the information segmentation module 10 is further configured to obtain anatomical connection relationships based on joints according to the human limb structure, and to segment the initial joint information into target structural regions according to the anatomical connection relationships, wherein the target structural regions include upper limb regions, lower limb regions and trunk regions; and to reconstruct the joint node sequence within the target structural regions to construct a limb substructure diagram;

[0176] Temporal variation features are extracted from the limb substructure diagram to obtain gait limb skeleton information.

[0177] This application provides a discrete tag-based emotion information recognition device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the discrete tag-based emotion information recognition method in the above embodiment 1.

[0178] The following is for reference. Figure 5This document illustrates a structural schematic diagram of a discrete tag-based emotion recognition device suitable for implementing embodiments of this application. The discrete tag-based emotion recognition device in this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The illustrated emotion information recognition device based on discrete tags is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0179] like Figure 5 As shown, the discrete tag-based emotion recognition device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the discrete tag-based emotion recognition device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the discrete tag-based emotion recognition device to communicate wirelessly or wiredly with other devices to exchange data. Although discrete tag-based emotion recognition devices with various systems are shown in the figures, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0180] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0181] The discrete label-based emotion recognition device provided in this application, employing the discrete label-based emotion recognition method described in the above embodiments, can solve the technical problem that existing gait emotion recognition methods cannot effectively model high-order emotion semantic information in self-supervised learning. Compared with the prior art, the beneficial effects of the discrete label-based emotion recognition device provided in this application are the same as those of the discrete label-based emotion recognition method provided in the above embodiments, and other technical features of this discrete label-based emotion recognition device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0182] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0183] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0184] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the emotion information recognition method based on discrete tags in the above embodiments.

[0185] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), Erasable Programmable Read Only Memory (EPROM), optical fiber, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0186] The aforementioned computer-readable storage medium may be included in a discrete tag-based emotion recognition device; or it may exist independently and not assembled into a discrete tag-based emotion recognition device.

[0187] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by a discrete label-based emotion recognition device, the discrete label-based emotion recognition device performs the following actions: re-divides initial joint information according to human limb structure to obtain gait limb skeleton information; generates initial discrete labels based on the gait limb skeleton information; performs loss calculation on the gait limb skeleton information, the initial joint information, and the initial discrete labels, and optimizes the initial discrete labels based on the loss calculation results to obtain target discrete labels; and performs multi-scale semantic alignment based on the target discrete labels to obtain multi-scale emotion information, thereby completing the discrete label-based emotion recognition.

[0188] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0189] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0190] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0191] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described discrete label-based emotion information recognition method. This addresses the technical problem that existing gait emotion recognition methods cannot effectively model high-order emotion semantic information in self-supervised learning. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the discrete label-based emotion information recognition method provided in the above embodiments, and will not be elaborated upon here.

[0192] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the discrete tag-based emotion information recognition method described above.

[0193] The computer program product provided in this application can solve the technical problem that existing gait emotion recognition methods cannot effectively model high-order emotional semantic information in self-supervised learning. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the discrete label-based emotion information recognition method provided in the above embodiments, and will not be repeated here.

[0194] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for identifying sentiment information based on discrete labels, characterized in that, The method includes: The initial joint information is re-divided based on the human limb structure to obtain gait limb skeleton information; Initial discrete labels are generated based on the gait limb skeletal information; The gait limb skeleton information, the initial joint information, and the initial discrete labels are used to calculate the loss, and the initial discrete labels are optimized based on the loss calculation results to obtain the target discrete labels; Multi-scale semantic alignment is performed based on the target discrete labels to obtain multi-scale sentiment information, thereby completing sentiment information recognition based on discrete labels.

2. The method as described in claim 1, characterized in that, The step of performing loss calculations on the gait limb skeleton information, the initial joint information, and the initial discrete labels, and optimizing the initial discrete labels based on the loss calculation results to obtain the target discrete labels includes: The number of limb parts, the initial prediction result, and the uniformly distributed category data are obtained, wherein the initial prediction result is the result of the previous model prediction. The gait limb skeletal information and the initial joint information are classified using a nearest neighbor algorithm to obtain a consistency score; The supervision loss is calculated based on the number of limb parts, the consistency score, the initial discrete labels, the initial prediction results, and the uniformly distributed category data. The process loss is calculated based on the consistency score, the initial prediction result, and the initial discrete label. The classification loss is calculated based on the supervision loss and the process loss, and the initial discrete label is optimized based on the classification loss to obtain the target discrete label.

3. The method as described in claim 2, characterized in that, The step of calculating the supervision loss based on the number of limb parts, the consistency score, the initial discrete labels, the initial prediction results, and the uniformly distributed category data includes: Calculate the cross-entropy loss based on the initial discrete labels and the initial prediction results; Calculate the divergence loss based on the uniformly distributed category data and the initial prediction results; The supervision loss is calculated based on the number of limb parts, the cross-entropy loss, and the divergence loss.

4. The method as described in claim 1, characterized in that, The step of generating initial discrete labels based on the gait limb skeletal information includes: Cluster the gait limb skeleton information to generate the current hard discrete label; Encoding calculations are performed based on the current hard discrete label to generate an initial discrete label.

5. The method as described in claim 1, characterized in that, The step of performing multi-scale semantic alignment based on the discrete labels to obtain multi-scale sentiment information, thereby completing the sentiment information recognition based on discrete labels, includes: Obtain skeleton nodes, body part nodes, and limb part nodes; Based on the discrete labels, joint information, limb information, and motion information are obtained; The mean square error is calculated based on the skeleton node, the body part node, the limb part node, the joint information, the limb information, and the motion information to obtain the mean square error calculation result for multiple scales. Then, the semantic alignment of multiple scales is performed based on the mean square error calculation result to obtain semantic information for multiple scales. Calculate the inter-scale semantic loss based on the skeleton nodes, body part nodes, limb part nodes, joint information, limb information, and motion information; The encoder parameters are updated based on the multi-scale semantic information and the inter-scale semantic loss, and multi-scale sentiment information is obtained based on the updated encoder parameters to complete the sentiment information recognition based on discrete labels.

6. The method as described in claim 5, characterized in that, The step of calculating the inter-scale semantic loss based on the skeleton nodes, body part nodes, limb part nodes, joint information, limb information, and motion information includes: Calculate the average value of inter-scale information based on the joint information, the limb information, and the motion information; The mean squared error is calculated based on the skeleton node, the body part node, the limb part node, and the average value of the inter-scale information to obtain the inter-scale mean squared error calculation result, and the inter-scale semantic loss is obtained based on the inter-scale mean squared error calculation result.

7. The method as described in claim 1, characterized in that, The step of re-dividing the initial joint information based on the human limb structure to obtain gait limb skeletal information includes: Based on the structure of the human limbs, the anatomical connection relationship based on the joints is obtained, and the initial joint information is divided into target structural regions according to the anatomical connection relationship. The target structural regions include the upper limb region, the lower limb region, and the trunk region. The joint node sequence within the target structural region is reconstructed to construct a limb substructure diagram; Temporal variation features are extracted from the limb substructure diagram to obtain gait limb skeleton information.

8. A device for recognizing emotion information based on discrete tags, characterized in that, The device includes: The information segmentation module is used to re-segment the initial joint information according to the human limb structure to obtain gait limb skeleton information; The tag generation module is used to generate initial discrete tags based on the gait limb skeleton information; The loss calculation module is used to perform loss calculation on the gait limb skeleton information, the initial joint information and the discrete labels, and optimize the initial discrete labels according to the loss calculation results to obtain the target discrete labels; The information recognition module is used to perform multi-scale semantic alignment based on the target discrete label to obtain multi-scale sentiment information, so as to complete the sentiment information recognition based on discrete label.

9. A sentiment information recognition device based on discrete tags, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the emotional information recognition method based on discrete tags as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the emotion information recognition method based on discrete tags as described in any one of claims 1 to 7.

Images