Human-object interaction classification method based on category index learning

Abstract

The application provides a human-object interaction classification method based on category index learning, which has the following characteristics: step S1, obtaining image features and human-object interaction instance features according to human-object interaction images; step S2, obtaining updated category indexes according to the image features and the category indexes; step S3, obtaining image classification scores according to the updated category indexes; step S4, obtaining instance classification scores according to the image classification scores; and steps S5-S7, obtaining interaction recognition results of the human-object interaction images according to the image classification scores and the instance classification scores. In summary, the method can improve the recognition accuracy of human-object interaction classification.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision and human-object interaction detection, and specifically relates to a human-object interaction classification method based on category index learning. Background Technology

[0002] Human action recognition has been a crucial research branch in computer vision for decades. An important subtask within human action recognition not only requires recognizing human actions but also locating the interaction between the person and surrounding objects; this task, known as Human-Object Interaction (HOI) detection, is key to understanding the semantics of human actions. HOI detection is a vital core technology for in-depth scene understanding and visual cognition, with broad application prospects and significant market demand in fields such as security monitoring, medical education, motion analysis, and autonomous driving.

[0003] The human-object interaction detection task can be divided into two sub-tasks: human-object pair detection and interaction classification. The former focuses on detecting people and objects in an image and determining whether they have an interaction relationship, which is highly related to the object detection task; the latter focuses on classifying the interaction categories of human-object pairs, requiring the model to understand a wide variety of interaction semantics, which is very challenging.

[0004] Existing human-object interaction detection methods are primarily based on deep learning techniques, and the transformer architecture has also seen widespread use in recent years. Therefore, existing human-object interaction detection technologies can be broadly categorized into three types based on their workflow and network structure: two-stage methods that first detect human-object pairs, then extract and classify human-object interaction features; single-stage methods that extract features from both human-object pair detection and interaction classification subtasks in parallel and merge the results of the two subtasks through post-processing; and end-to-end methods based on the transformer architecture that output human-object interaction detection results end-to-end.

[0005] In addressing the subtask of interaction classification, existing methods largely focus on extracting better interaction features for instance classification. For example, Ulutan et al. attempted to use graph neural networks to model the contextual information of human-object interactions; Zhang et al. utilized attention mechanisms to extract semantically relevant information from the scene; Yuan et al. used visual-linguistic cross-modal information to assist in understanding interaction semantics; and Liao et al. made improvements based on the transformer architecture, decoupling the two subtasks to ensure that features from the interaction classification task were not used to interfere with the detection of human and object features. However, these methods use a single weight vector as the class center for each category in interaction classification, leaving significant room for improvement in the accuracy of interaction classification. Summary of the Invention

[0006] This invention is made to solve the above-mentioned problems, and aims to provide a human-object interaction classification method based on category index learning.

[0007] This invention provides a human-object interaction classification method based on category index learning. It is characterized by obtaining the interaction recognition result of a human-object interaction image based on the human-object interaction image and K category indices generated from different image categories. The method includes the following steps: Step S1, inputting the human-object interaction image into a baseline human-object pair detector to obtain image features I and n human-object interaction instance features; Step S2, inputting image features I and the K category indices into a transformer network to obtain K updated category indices; Step S3, inputting the K updated category indices into an image classification score network to obtain the corresponding image category for each human-object interaction image. K image classification scores; Step S4, use the K updated category indices as classification weights for the corresponding image categories, and calculate the cosine similarity with the n person-object interaction instance features to obtain the instance classification scores of each person-object interaction instance feature on the K image categories; Step S5, sort the K image classification scores from largest to smallest, select the top κ image classification scores as retained scores, and the image categories corresponding to the κ retained scores as predicted image categories; Step S6, for each person-object interaction instance feature, sequentially select the instance classification scores of the person-object interaction instance feature on the predicted image categories to construct an initial hard score set, using κ learnable parameters [τ1,…,τ] κ The initial hard score set is adjusted to obtain the hard score set of human-object interaction instance features; in step S7, the hard score set of human-object interaction instance features and the κ retained scores are fused with soft scores to obtain the soft score set of human-object interaction instance features. The soft score set of all human-object interaction instance features of the human-object interaction image is the interaction recognition result of the human-object interaction image.

[0008] The human-object interaction classification method based on category index learning provided by this invention may also have the following features: In step S1, the baseline human-object pair detector includes a feature extraction network and a human-object pair detection network. Step S1 includes the following sub-steps: Step S1-1, input the human-object interaction image into the feature extraction network to obtain image feature I; Step S1-2, input the image feature I into the human-object pair detection network to obtain n human-object interaction instance features.

[0009] The human-object interaction classification method based on category index learning provided by the present invention may also have the following feature: wherein, in step S2, the transformer network is a multi-layer transformer decoder.

[0010] The human-object interaction classification method based on category index learning provided by this invention may also have the following feature: when training the image classification score network in step S3 using training samples, the loss function used is the image classification loss function L. img Image classification loss function L img The calculation formula is as follows: p′ k =max(p k -m,0), where K is the total number of image categories in the training samples, γ+, γ- and m are hyperparameters, p k y is the image classification score obtained based on the human-object interaction images input from the training samples. k Let y be the label of the k-th image category in the training samples. k =1 indicates that the k-th image category exists in the training human-object interaction images, y k =0 indicates that the k-th image category does not exist in the training human-object interaction images.

[0011] The human-object interaction classification method based on category index learning provided by this invention can also have the following feature: wherein the classification score p of the k-th image in the training samples k The calculation formula is as follows: p k =sigmoid(FC(Q′) k In the formula, Q′ k This represents the updated class index corresponding to the k-th class index during training. FC() is a fully connected layer operation, and sigmoid() is a sigmoid operation.

[0012] The human-object interaction classification method based on category index learning provided by this invention may also have the following feature: wherein, in step S4, the formula for calculating the instance classification score is: In the formula s i，k Let Q' be the instance classification score of the i-th person-object interaction instance feature in the k-th image category, sigmoid() is the sigmoid operation, and Q' is the instance classification score of the i-th person-object interaction instance feature in the k-th image category. k For the k-th updated category index, F i Let i be the feature of the i-th person-object interaction instance.

[0013] The human-object interaction classification method based on category index learning provided by this invention may also have the following feature: wherein, in step S6, the initial hard score set is represented as follows: In the formula F is the initial hard score set containing κ initial hard scores for the features of the i-th person-object interaction instance. i Let i be the features of the i-th person-object interaction instance. The updated category index corresponding to the first retained score after sorting. The hard score set is represented as follows, where the updated category index corresponds to the κ-th retained score after sorting is used: In the formula Let be the hard score set of the features of the i-th person-object interaction instance.

[0014] The human-object interaction classification method based on category index learning provided by this invention may also have the following feature: wherein, in step S7, the soft fraction set is represented as: In the formula Let i be the soft score set of features of the i-th person-object interaction instance. Let be the first value in the hard score set of the features of the i-th person-object interaction instance. The first score to retain after sorting. Let be the κ-th value in the hard score set of the features of the i-th person-object interaction instance. This is the κ-th retained score after sorting.

[0015] The role and effect of invention

[0016] According to the category index-based human-object interaction classification method of the present invention, by using category indices generated from different image categories as dynamic class centers instead of the traditional single weight vector, the instance classification scores are corrected, thereby achieving better interaction recognition accuracy. Therefore, the category index-based human-object interaction classification method of the present invention can improve the recognition accuracy of human-object interaction classification. Attached Figure Description

[0017] Figure 1 This is a schematic diagram illustrating the principle of the human-object interaction classification method based on category index learning in an embodiment of the present invention;

[0018] Figure 2 This is a flowchart illustrating the human-object interaction classification method based on category index learning in an embodiment of the present invention;

[0019] Figure 3 This is a schematic diagram of the interactive recognition results in an embodiment of the present invention. Detailed Implementation

[0020] To make the technical means, creative features, objectives and effects of this invention easier to understand, the following embodiments, in conjunction with the accompanying drawings, specifically illustrate the human-object interaction classification method based on category index learning of this invention.

[0021] Figure 1 This is a schematic diagram illustrating the principle of the human-object interaction classification method based on category index learning in an embodiment of the present invention.

[0022] The human-object interaction classification method based on category index learning in this embodiment includes: a baseline human-object interaction detector, image-level category index learning, and category index-based interaction classification. In the baseline human-object interaction detector, image features I are obtained by inputting human-object interaction images into a feature extraction network, and image features I are input into a human-object pair detection network to obtain human-object instance features F. i And the corresponding human bounding box, object bounding box (i.e., human / object bounding box), and object category. In image-level category index learning, the image features I and the initial category index Q are... k For example, inputting words like "ride," "lift," and "eat" into the transformer decoder yields the updated category index Q. k ', will update the category index Q k 'Through a fully connected layer, the corresponding image classification score p is obtained.' k In interactive classification based on category indexes, the updated category index Q will be... k 'and person-object instance features F i Cosine similarity calculation is performed to obtain the person-object instance feature F. i Instance classification score s corresponding to each category index i Then, compared with the corresponding image classification score p k Score fusion is performed to calculate the final instance classification score, which serves as the recognition result for the human-object interaction image. The loss functions used during training are used to calculate the image classification loss, instance classification loss, and other losses.

[0023] Figure 2 This is a flowchart illustrating the human-object interaction classification method based on category index learning in an embodiment of the present invention.

[0024] like Figure 2 As shown, the human-object interaction classification method based on category index learning of the present invention is used to obtain the interaction recognition result of the human-object interaction image based on the human-object interaction image and K category indices generated by different image categories, including the following steps:

[0025] Step S1: Input the human-object interaction image into the baseline human-object pair detector to obtain image features I∈R H×W×D Features of n person-object interaction instances. R indicates that the number of image features I is a real number, H and W are the length and width of the image feature, respectively, and D is the number of channels of the image feature.

[0026] In this embodiment, the baseline person-object pair detector is an existing person-object interaction detection method that removes the last interaction classification layer. The baseline person-object pair detector includes a feature extraction network, a person-object pair detection network, and a classification head and a detection head network, and includes the following sub-steps:

[0027] Step S1-1: Input the human-object interaction image into the feature extraction network to obtain image feature I.

[0028] The feature extraction network is either a convolutional neural network or a transformer network.

[0029] Steps S1-2 involve inputting image feature I into the human-object pair detection network to obtain n human-object interaction instance features. In this embodiment, the human-object pair detection network is a detection transformer decoder network.

[0030] In this embodiment, image feature I is input into the classification head and detection head network to obtain the human bounding box, object bounding box and object category of each human-object interaction instance feature.

[0031] Step S2: Input the image features I and K category indices into the transformer network to obtain K updated category indices.

[0032] The transformer network is a multi-layer transformer decoder.

[0033] In this embodiment, K category indices are used as network parameters of the transformer network. Each category index has a dimension of D, and the category index is represented as {Q}. k}∈R K×D The transformer network updates the K category indices based on image features I. The transformer network operates in parallel with the baseline human-object pair detector and does not contain a transformer encoder. The update process is as follows:

[0034] {Q′ k} = decoder({Q k},I),

[0035] In the formula, decoder() is a transformer decoder, {Q′ k} represents the updated category index.

[0036] Step S3: Input the K updated category indices into the image classification score network to obtain the K image classification scores for each image category corresponding to the human-object interaction image.

[0037] In this embodiment, the image classification score network includes K fully connected layers and a sigmoid operation. The K fully connected layers correspond to K updated category indices, and the process of obtaining K image classification scores is as follows:

[0038] p k =sigmoid(FC(Q′)k )),

[0039] In the formula p k Let represent the classification score of the k-th image, sigmoid() is the sigmoid operation, and FC() is a fully connected layer.

[0040] Step S4: Use the K updated category indices as the classification weights of the corresponding image categories, and calculate the cosine similarity with the n person-object interaction instance features in turn to obtain the instance classification score of each person-object interaction instance feature on the K image categories.

[0041] The formula for calculating the instance classification score is as follows:

[0042]

[0043] In the formula s i，k Let Q' be the instance classification score of the i-th person-object interaction instance feature in the k-th image category, sigmoid() is the sigmoid operation, and Q' is the instance classification score of the i-th person-object interaction instance feature in the k-th image category. k For the k-th updated category index, F i Let i be the feature of the i-th person-object interaction instance.

[0044] Feature F of the i-th person-object interaction instance i The set s of all instance classification scores across K image categories i Represented as:

[0045]

[0046] In the formula, Q′1 is the first updated category index, and Q′ K This is the index of the Kth updated category.

[0047] Step S5: Sort the K image classification scores from largest to smallest, select the top k image classification scores as retained scores, and use the image categories corresponding to the k retained scores as predicted image categories.

[0048] Step S6: For each person-object interaction instance feature, sequentially select the instance classification score of the person-object interaction instance feature on the predicted image category to construct an initial hard score set, using κ learnable parameters [τ1, ..., τ]. κ The initial hard score set is adjusted to obtain the hard score set of human-object interaction instance features. In this embodiment, there are κ learnable parameters [τ1, ..., τ2]. κ ] is part of the network parameters, which are randomly initialized and the final parameter values ​​are obtained after training the network using training samples.

[0049] The initial hard fraction set is represented as follows:

[0050]

[0051] In the formula F is the initial hard score set containing κ initial hard scores for the features of the i-th person-object interaction instance. i Let i be the features of the i-th person-object interaction instance. The updated category index corresponding to the first retained score after sorting. This is the updated category index corresponding to the κ-th retained score after sorting.

[0052] The hard fraction set is represented as follows:

[0053]

[0054] In the formula Let be the hard score set of the features of the i-th person-object interaction instance.

[0055] Step S7: Soft score fusion is performed on the hard score set of human-object interaction instance features and κ retained scores to obtain the soft score set of human-object interaction instance features. The soft score set of all human-object interaction instance features of the human-object interaction image is the interaction recognition result of the human-object interaction image.

[0056] The soft fraction set is represented as:

[0057]

[0058] In the formula Let i be the soft score set of features of the i-th person-object interaction instance. Let be the first value in the hard score set of the features of the i-th person-object interaction instance. The first score to retain after sorting. Let be the κ-th value in the hard score set of the features of the i-th person-object interaction instance. This is the κ-th retained score after sorting.

[0059] In this embodiment, the model is trained using training samples. The loss functions include: the baseline human-object interaction detector loss function and the image classification loss function L. img And the focus loss function. The final loss is obtained by summing the three loss functions using training samples, and the entire model is trained using the backpropagation algorithm.

[0060] The baseline human-object interaction detector loss function is used to train the baseline human-object interaction detector, calculating the human bounding box regression loss, the object bounding box regression loss, and the object classification loss, i.e., other losses.

[0061] Image classification loss function L imgThe image classification loss function, L, is used to train an image classification score network and calculate the image classification loss. It is an asymmetric focal loss function. img The calculation formula is as follows:

[0062]

[0063] p′ k =max(p k -m,0),

[0064] In the formula, K is the total number of image categories in the training samples, γ+, γ-, and m are all hyperparameters, and p k y is the image classification score obtained based on the human-object interaction images input from the training samples. k Let y be the label of the k-th image category in the training samples. k =1 indicates that the k-th image category exists in the training human-object interaction images, y k =0 indicates that the k-th image category does not exist in the training human-object interaction images.

[0065] Wherein, the classification score p of the k-th image in the training samples k The calculation formula is as follows:

[0066] p k =sigmoid(FC(Q′) k )),

[0067] In the formula Q′ k This represents the updated class index corresponding to the k-th class index during training. FC() is a fully connected layer operation, and sigmoid() is a sigmoid operation.

[0068] The focus function is used to train the generated instance classification scores and calculate the instance classification loss.

[0069] In this embodiment, the existing QPIC model and the model combining the QPIC model with the human-object interaction classification method based on category index learning of the present invention are used to perform interaction recognition tests on images in the public dataset HICO-DET to obtain interaction recognition results.

[0070] Figure 3 This is a schematic diagram of the interactive recognition results in an embodiment of the present invention.

[0071] like Figure 3As shown, the black dashed boxes represent the bounding boxes of the human body, and the black solid boxes represent the bounding boxes of the objects. The centers of the human body bounding boxes and the object bounding boxes are connected by a black solid line, indicating that the corresponding human and object form a human-object interaction pair. The left image, TP (true positive), represents the expected recognition result of the two models for classifying the human-object interaction in this image, i.e., detecting the correct instance "person sit on bus," including the human body bounding box, the object bounding box, the prediction probability of 0.15 for the existing QPIC model, and the prediction probability of 0.38 for the model combining the QPIC model with the present invention. The right image, FP (false positive), represents the undesired recognition result of the two models for classifying the human-object interaction in this image, i.e., detecting the incorrect instance "person board bus," including the human body bounding box, the object bounding box, the prediction probability of 0.29 for the QPIC model, and the prediction probability of 0.07 for the model combining the QPIC model with the present invention. It is evident that the method of the present invention can improve the probability of existing models recognizing the desired human-object interaction classification and reduce the probability of existing models recognizing the undesired human-object interaction classification.

[0072] This embodiment combines the existing QPIC model, SCG model, and GEN-VLKT model with the human-object interaction classification method based on category index learning of this invention, resulting in three models: QPIC+Invention Model, SCG+Invention Model, and GEN-VLKT+Invention Model. Using the public datasets HICO-DET and V-COCO as test datasets, the mean accuracy (mAP) of each model was calculated for all 600 categories (Full), 138 rare categories (Rare), and 462 common categories (Non-Rare) in the HICO-DET dataset. The mean accuracy (mAP) of each model in the V-COCO dataset was also calculated for the prediction results (S1) when the object labeling is missing and the prediction results (S2) when the object labeling is missing and the object position is not considered during evaluation. The parameter count and inference speed (FPS) of each model are shown in the table below.

[0073]

[0074]

[0075] The first column of the table lists the models; the second to fourth columns show the mAP (%) of each model in the Full, Rare, and Non-Rare cases of the HICO-DET dataset, respectively; the fifth and sixth columns show the AP (%) of each model in the S1 and S2 cases of the V-COCO dataset, respectively; the seventh column shows the number of parameters for each model; and the eighth column shows the inference speed for each model. For example, the cell in the second row and second column shows that the mAP (%) of the QPIC model in the Full case of the HICO-DET dataset is 28.93. According to the table above, the human-object interaction classification method based on category index learning of this invention can use QPIC, SCG, and GEN-VLKT models as baselines, and it outperforms the original models in average accuracy across various cases on both test datasets. Furthermore, the increase in parameter size is relatively small, and the decrease in inference speed is also relatively small. Therefore, the human-object interaction classification method based on category index learning of this invention can effectively improve the recognition performance of existing models in human-object interaction classification.

[0076] The average precision (mAP) of existing models that perform well on the HICO-DET dataset in this embodiment, namely QPIC model, CDN model, HQM model, IF model, GEN-VLKT model (ResNet50), GEN-VLKT model (ResNet101), and BodyPartMap model, and the GEN-VLKT models combined with the human-object interaction classification method based on category index learning of this invention, namely GEN-VLKT+Invention Model (ResNet50) and GEN-VLKT+Invention Model (ResNet101), are shown in the table below:

[0077]

[0078] The first column of the table lists the various models, the second column lists the backbone network for each model, and the third to fifth columns show the mAP (%) for each model in the Full, Rare, and Non-Rare cases of the HICO-DET dataset, respectively. For example, the cell in the ninth row and third column shows that the GEN-VLKT model with a ResNet50 backbone network, combined with the model of this invention, has an mAP (%) of 35.36 in the Full case of the HICO-DET dataset. According to the table above, the human-object interaction classification method based on category index learning of this invention can improve the accuracy of existing models in interaction classification on the HICO-DET dataset, and the combined model can achieve leading results.

[0079] The average precision (AP) of existing models that perform well on the V-COCO dataset in this embodiment, namely HQM model, CDN model, GEN-VLKT model (ResNet50), GEN-VLKT model (ResNet101), DT model, and STIP model, and the GEN-VLKT models combined with the category index learning-based human-object interaction classification method of this invention, namely GEN-VLKT+Invention Model (ResNet50) and GEN-VLKT+Invention Model (ResNet101), are shown in the table below:

[0080]

[0081]

[0082] The first column of the table lists the various models, the second column lists the backbone network for each model, and the third and fourth columns show the AP (%) for each model in S1 and S2 cases of the V-COCO dataset, respectively. For example, the cell in the eighth row and third column indicates that the GEN-VLKT model with a ResNet50 backbone network, combined with the model of this invention, has an AP (%) of 66.4 in S1 case of the V-COCO dataset. According to the table above, the human-object interaction classification method based on category index learning of this invention can improve the accuracy of existing models in interaction classification on the V-COCO dataset, and the combined model can achieve leading results.

[0083] The role and effect of the embodiments

[0084] According to the human-object interaction classification method based on category index learning involved in this embodiment, the category index generated by different image categories is used as a dynamic class center to replace the traditional single weight vector, thereby correcting the instance classification score and achieving better interaction recognition accuracy. In summary, this method can improve the recognition accuracy of human-object interaction classification.

[0085] The above embodiments are preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention.

Claims

1. A human-object interaction classification method based on category index learning, characterized in that, The method for obtaining the interaction recognition result of the human-object interaction image based on the human-object interaction image and K category indices generated from different image categories includes the following steps: Step S1: Input the human-object interaction image into the baseline human-object pair detector to obtain image feature I and n human-object interaction instance features; Step S2: Input the image feature I and the K category indices into the transformer network to obtain K updated category indices; Step S3: Input the K updated category indices into the image classification score network to obtain K image classification scores for each image category corresponding to the human-object interaction image; Step S4: Use the K updated category indices as the classification weights of the corresponding image categories, and calculate the cosine similarity with the n person-object interaction instance features in turn to obtain the instance classification score of each person-object interaction instance feature on the K image categories; Step S5: Sort the K image classification scores from largest to smallest, select the top κ image classification scores as retained scores, and use the image categories corresponding to the κ retained scores as predicted image categories; Step S6: For each human-object interaction instance feature, sequentially select the instance classification score of the human-object interaction instance feature on the predicted image category to construct an initial hard score set, using κ learnable parameters [τ1,…,τ]. κ The initial hard score set is adjusted to obtain the hard score set of the human-object interaction instance features; Step S7: Soft score fusion is performed on the hard score set of the human-object interaction instance features and the κ retained scores to obtain the soft score set of the human-object interaction instance features. The soft score set of all the human-object interaction instance features of the human-object interaction image is the interaction recognition result of the human-object interaction image.

2. The human-object interaction classification method based on category index learning according to claim 1, Its features are: In step S1, the baseline person-object pair detector includes a feature extraction network and a person-object pair detection network, and step S1 includes the following sub-steps: Step S1-1: Input the human-object interaction image into the feature extraction network to obtain the image feature I; Step S1-2: Input the image feature I into the human-object pair detection network to obtain the n human-object interaction instance features.

3. The human-object interaction classification method based on category index learning according to claim 1, characterized in that: in, In step S2, the transformer network is a multi-layer transformer decoder.

4. The human-object interaction classification method based on category index learning according to claim 1, characterized in that: in, When training the image classification score network in step S3 using training samples, the loss function used is the image classification loss function L. img The image classification loss function L img The calculation formula is as follows: p′ k =max(p k -m,0), In the formula, K is the total number of image categories in the training samples, γ+, γ-, and m are all hyperparameters, and p k The image classification score y is obtained based on the human-object interaction images input from the training samples. k Let y be the label of the k-th image category in the training samples. k =1 indicates that the k-th image category exists in the training human-object interaction image, y k =0 indicates that the k-th image category does not exist in the training human-object interaction images.

5. The human-object interaction classification method based on category index learning according to claim 4, characterized in that: in, The classification score p of the k-th image in the training samples k The calculation formula is as follows: p k =sigmoid(FC(Q′ k )), In the formula Q′ k The updated category index corresponds to the k-th category index during training. FC() is a fully connected layer operation, and sigmoid() is a sigmoid operation.

6. The human-object interaction classification method based on category index learning according to claim 1, characterized in that: in, In step S4, the formula for calculating the instance classification score is: In the formula s i，k Let Q' be the instance classification score of the i-th person-object interaction instance feature in the k-th image category, sigmoid() is the sigmoid operation, and Q' is the instance classification score of the i-th person-object interaction instance feature in the k-th image category. k For the k-th updated category index, F i Let i be the feature of the i-th person-object interaction instance.

7. The human-object interaction classification method based on category index learning according to claim 1, characterized in that: in, In step S6, the initial hard fraction set is represented as follows: In the formula F is the initial hard score set containing κ initial hard scores for the features of the i-th person-object interaction instance. i Let i be the features of the i-th person-object interaction instance. The updated category index corresponding to the first retained score after sorting. This is the updated category index corresponding to the κ-th retained score after sorting. The hard fraction set is represented as follows: In the formula Let be the hard score set of the features of the i-th person-object interaction instance.

8. The human-object interaction classification method based on category index learning according to claim 1, characterized in that: in, In step S7, the soft fractional set is represented as: In the formula Let i be the soft score set of features of the i-th person-object interaction instance. Let be the first value in the hard score set of the features of the i-th person-object interaction instance. The first score to retain after sorting. Let k be the k-th value in the hard score set of the features of the i-th person-object interaction instance. This is the κ-th retained score after sorting.

Images