NEW FRAMEWORK FOR TIME LOCALIZATION OF ACTIONS WITH FEW RECORDS

Abstract

System (102) comprising: a memory (110) that stores computer-executable components; and a processor (108) that is functionally connected to the memory and that executes the computer-executable components stored in the memory, wherein the computer-executable components comprise: a graph component (116) that generates a graph (402) that models a lemma (104) of temporal action classifications, such that nodes of the graph correspond to respective temporal action classifications in the lemma and edges of the graph correspond to similarities between the respective temporal action classifications;and a convolution component (120) that performs a convolution on the graph so that the nodes output respective agreement values ​​indicating the degrees of agreement between the respective temporal action classifications and an action to be classified, wherein the system further comprises an instantiation component (118) that, based on a proposed feature vector (302) representing the action to be classified, inputs respective input vectors (602) into the nodes, wherein the respective temporal action classifications correspond to respective example feature vectors (202);and the respective input vectors are concatenations of the respective example feature vectors and the proposed feature vector representing the action to be classified, wherein the action to be classified is an action displayed in an untrimmed input video, and wherein the action to be classified is found to belong to a temporal action classification with a highest match value.

Description

BACKGROUNDThe present disclosure relates to localization of actions in machine vision and, more particularly, to temporal localization of actions with few shots based on graph convolution networks. Temporal action localization includes receiving an untrimmed video, locating (e.g., identifying start and end times) an action displayed in the untrimmed video, and classifying the action (e.g., identifying the action as running, jumping, casting, etc.). Conventional techniques for time-based action localization require large amounts of training data, whose acquisition can be very time-consuming and expensive. A few shot temporal action localization solves this problem by learning how actions are classified based on only a few examples (e.g., a small number of examples). Most existing systems for time-based action localization with few recordings use a model-agnostic meta-learning (MAML) framework. Other existing systems for time-based action localization with few shots use learning sequence matching networks. In any case, systems / methods are advantageous which can achieve a time-based action localization with few recordings with higher accuracy / precision than the existing systems / methods for time-based action localization with few recordings.The scientific publication "Graph Convolutional Networks for Temporal Action Localization" by R. Zeng et al., published on 07.09.2019 at https: / / arxiv.org / abs / 1909.03252 addresses the problem that many modern action localization systems process each action proposal individually without explicitly utilizing their relationships during learning. To do this, the authors propose to use the relationships between the suggestions using Graph Convolutional Networks (GCNs). First, an action suggestion graph is created in which each suggestion is represented as nodes and the relationships between two suggestions are represented as edges. Two types of relationships are used here: one for acquiring the context information for each proposal and one for characterizing the correlations between different actions. Subsequently, the GCNs are applied to the graph to model the relationships between different suggestions and learn representations for action classification and localization. Experimental results are presented, which indicate that the approach presented outgoes a previous method on THUMOS14. To verify the effectiveness of modeling relationships between action suggestions, results of augmentation attempts on ActivityNet are presented.The scientific publication "Edge-labeling graph neural network for few-shot learning" by J. Kim et al., published 04.05.2019 at https: / / arxiv.org / abs / 1905.01436 describes an edge-characterizing graph neural network (EGNN) that adjusts a deep neural network on the edge-characterizing graph for few-shot learning. Unlike previous graph neural network (GNN) approaches for Few-Shot learning that are based on the node label framework that implicitly models similarity within clusters and dissimilarity between clusters, the proposed EGNN learns to predict the edge labels instead of the node labels in the graph. This is intended to allow explicit clustering to be developed by iteratively updating the edge descriptors using both similarity within the clusters and dissimilarity between the clusters directly. The suitability of the network for execution with a different number of classes without retraining and its extensibility to perform a transducing inference are also investigated. The parameters of the EGNN are learned by episodic training with an edge labeling loss to obtain a generalizable model for unknown problems with small amounts of data. Performance evaluation is performed on supervised as well as semi-supervised few shot image classification tasks with two benchmark data sets.CN 1 10 362 715 A describes a graph convolution network based method for temporal location of raw video action, comprising: obtaining a confidence score whether or not each video clip in the annotated raw video is an action; merging the video clips using a "double threshold method" to obtain an action proposal of the video; calculating an overlap rate between the action proposal and the true value; calculating the overlap rate between the action proposal and the true value and assuming the action proposal whose loU value satisfies the conditions as a node in the graph model; calculating the neighbor matrix to define the similarity between any two nodes in the graph model; obtaining the interaction information between any node and all other nodes in the graph model and updating the feature representation of the nodes in the graph model; and performing the time-limited regression and the action classification of the action proposal and obtaining the result of the time-localization of the action.CN 108 229 355 A describes a method and apparatus for behavior recognition, an electronic device, a computer storage medium, and a program, the method comprising: performing recognition of human key points on at least one frame of a video image; obtaining a plurality of human key points of the at least one frame of the video image; obtaining, based on feature information of the plurality of human key points of the at least one frame of the video image and mapping information of the plurality of human key points, a behavior recognition result of the at least one frame of the video image.SUMMARYA summary is provided to provide a basic understanding of one or more embodiments of the invention. This summary is not intended to identify key elements or key elements, or to outline any scope of the specific embodiments or a scope of the claims. Its sole purpose is to present concepts in a simplified form in anticipation of the more detailed description given below. In one or more embodiments described herein, units, systems, computer implemented methods, apparatus, and / or computer program products are described that enable temporal action location with few shots based on graph convolution networks.According to one or more embodiments, a system is provided. The system may include memory capable of storing computer-executable components. The system may further include a processor operatively coupled to the memory and operable to execute the computer-executable components stored in the memory. In various embodiments, the computer executable components may include a graph component that may generate a graph. In various cases, the graph may model an auxiliary set of temporal action classifications. According to one or more aspects, nodes of the graph may correspond to respective temporal action classifications in the auxiliary set. According to various aspects, edges (edges) of the graph may correspond to similarities between the respective temporal action classifications. In various embodiments, the computer executable components may further include a convolution component that may perform convolution on the graph. In various cases, the nodes of the graph after convolution may output corresponding match values. In various cases, the respective match values may indicate degrees of match between the respective temporal action classifications and an action to be classified. In one or more embodiments, the computer-executable components may further include an instantiation component that may input corresponding input vectors to the nodes. In various cases, the respective input vectors may be based on a proposed feature vector representing the action to be classified. In one or more embodiments, the respective temporal action classifications may correspond to respective example feature vectors, and the respective input vectors may be concatenations of the respective example feature vectors, where the proposed feature vector represents the action to be classified. In one or more embodiments, the respective example feature vectors may be generated via a first gated recurrent unit (first gated recurrent unit) based on respective single shot auxiliary videos showing examples of the respective temporal action classifications. In various cases, the proposed feature vector may be generated via a second gated recurrent unit based on an untrimmed input video showing the action to be classified. In one or more embodiments, the similarities may be calculated based on cosine similarities between respective example feature vectors of respective temporal action classifications.According to one or more embodiments, the system described above may be implemented as a computer implemented method. The method implemented by a computer may include generating, by a unit operatively connected to a processor, the graph modeling the subset of temporal action classifications and performing, by the unit, a convolution on the graph to output the respective match values.According to one or more embodiments, the system described above may be implemented as a computer program product to enable temporal action localization with few shots. The computer program product may comprise a computer readable storage medium having program instructions embodied thereon. The program instructions may be executable by a processing component and thus may cause the processing component to generate the graph modeling the auxiliary set of temporal action classifications and perform a convolution on the graph.One or more embodiments of the claimed innovation can produce more accurate / precise results than existing systems / methods for temporal action localization with few recordings. As mentioned above, existing systems / techniques for time-based, low-shot action localization use either MAML frameworks or learning sequence matching networks. Although such systems / methods may enable temporal action localization with few shots, they compare a proposed feature to be classified independently with example features in an auxiliary set (e.g., comparing a proposed feature to a running example, comparing the proposed feature to a jump example, comparing the proposed feature to a throw example, etc.). Such systems / methods do not consider the relationships between the examples in the auxiliary set (e.g., relationship between the example of travel and the example of jump, relationship between the example of jump and the example of throw, relationship between the example of throw and the example of travel, etc.). Various embodiments of the claimed invention may use these relationships within the kit to improve the effectiveness of a few shot temporal action localization. In particular, one or more embodiments of the claimed innovation may generate a graph modelling an auxiliary sentence. The auxiliary set may include one or more single shot auxiliary video chips (or, in some embodiments, auxiliary images), each single shot auxiliary video chip showing an example of a corresponding / respective temporal action classification. For example, the auxiliary set may include a first single shot auxiliary video slide showing an example of a person in progress (e.g., a time-based progress action classification), a second single shot auxiliary video slide showing an example of a jumping person (e.g., a time-based jump action classification), a third single shot auxiliary video slide showing an example of a person dropping an object (e.g., a time-based drop action classification), and so forth. Nodes of the graph may each correspond to the temporal action classifications in the auxiliary set (e.g., a first node corresponding to the walking classification, a second node corresponding to the jumping classification, a third node corresponding to the throwing classification, and so forth). Edges of the graph may correspond to similarities between the temporal action classifications (e.g., an edge between the first and second nodes may correspond to a similarity value between the run classification and the jump classification, an edge between the second and third nodes may correspond to a similarity value between the jump classification and the throw classification, an edge between the first and third nodes may correspond to a similarity value between the run classification and the throw classification, and so forth). In various embodiments, the similarities may be based on cosine similarity values between vectors representing the examples of the respective action classifications. Then, convolution of the graph may be performed over a trained graph convolution network to convey messages that may cause each node to output a match value. The match score of each node may indicate a degree of match between the action classification corresponding to that node and an action to be classified (e.g., an action displayed in an untrimmed input video). In various cases, the action classification having the highest match value may be determined to classify the action. Experimental results, measured with mean average precision (mAP), show that enabling temporal action localization via graph convolution networks offers significantly higher accuracy / precision than existing systems / methods for temporal action localization with few recordings on the same dataset.Overall, various embodiments of the claimed innovation may use relationships between examples in an auxiliary sentence to improve the effectiveness of a temporal action localization with few recordings. By modeling the auxiliary set as a graph and convoluting the graph over a trained graph convolute network, various embodiments of the claimed innovation can provide more accurate / precise results of temporal action localization than existing systems / methods for temporal action localization with few shots.DESCRIPTION OF THE DRAWINGSEmbodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which: FIG. 1 illustrates a block diagram of an example, non-limiting system that enables low-shot temporal action localization over graph convolution networks, in accordance with one or more embodiments described herein. FIG. 2 illustrates a block diagram of an example, non-limiting system having example feature vectors that enables low-shot temporal action localization over graph convolution networks, in accordance with one or more embodiments described herein. FIG. 3 illustrates a block diagram of an example, non-limiting system having a proposed feature vector that enables temporal action localization with few recordings over graph convolution networks, in accordance with one or more embodiments described herein. FIG. 4 illustrates a block diagram of an example non-limiting system including a graph that enables temporal action localization with few recordings over graph convolution networks, in accordance with one or more embodiments described herein. FIG. 5 illustrates a diagram of an example, non-limiting graph that enables temporal action localization with few recordings over graph convolution networks, in accordance with one or more embodiments described herein. FIG. 6 illustrates a block diagram of an example non-limiting input vector system that enables temporal action localization with few recordings over graph convolution networks, in accordance with one or more embodiments described herein. FIG. 7 illustrates a diagram of an example non-limiting graph with input concatenations that enables temporal action localization with few recordings over graph convolution networks, according to one or more embodiments described herein. FIG. 8 illustrates a block diagram of an example, non-limiting system including a graph convolution network that enables low-shot temporal action localization across graph convolution networks, in accordance with one or more embodiments described herein. FIG. 9 illustrates a diagram of an example non-limiting convoluted graph that enables temporal action localization with few recordings across graph convolution nets, in accordance with one or more embodiments described herein. FIG. 10 illustrates a high-level block diagram of an example, non-limiting system that enables low-shot temporal action localization via graph convolution networks, in accordance with one or more embodiments described herein. FIG. 11 illustrates a table of example, non-limiting results of a few shot temporal action localization over graph convolution networks according to one or more embodiments described herein. FIG. 12 illustrates a flow diagram of an example, non-limiting, computer-implemented method that enables low-shot temporal action localization over graph convolution networks, in accordance with one or more embodiments described herein. FIG. 13 illustrates a flow diagram of an example, non-limiting, computer-implemented method of input vectors that enables temporal action localization with few recordings over graph convolution networks, in accordance with one or more embodiments described herein. FIG. 14 illustrates a flow diagram of an example, non-limiting, computer-implemented method with input concatenations that enables low-shot temporal action localization over graph convolution networks, in accordance with one or more embodiments described herein. FIG. 15 illustrates a flow diagram of an example, non-limiting, computer-implemented gated recurrent unit method that enables low-shot temporal action localization over graph convolution networks, in accordance with one or more embodiments described herein. FIG. 16 illustrates a block diagram of an example, non-limiting operating environment in which one or more embodiments described herein may be enabled.DETAILED DESCRIPTIONThe following detailed description is merely illustrative and is not intended to limit embodiments and / or application or uses of embodiments. Furthermore, there is no intention of binding to any information expressly or implicitly indicated in the preceding background and summary sections or in the detailed description section.One or more embodiments will be described below with reference to the drawings, wherein like reference numerals are used throughout to designate like elements. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a more thorough understanding of the one or more embodiments. Clearly, however, in various instances, the one or more embodiments may be practiced without these specific details.Temporal action localization is a branch of machine learning and / or machine vision, receiving an untrimmed input video (e.g., a video slide), locating (e.g., identifying a start time and a stop time associated with) an action shown in the untrimmed input video, and classifying the action (e.g., determining that the action shows a person, animal, and / or other entity that is running, jumping, object, swimming, playing fullyball, climbing a tree, taking food, driving a vehicle, sleeping, looking at a movie, playing an instrument, feeling, deering, figking, falling, The activity is otherwise involved in an identified activity and / or inactivity, and so forth). In principle, temporal action localization requires large amounts of training data, the acquisition of which can be time-consuming and expensive. Temporal action localization with few shots solves this problem by learning to classify actions based on only a few examples (therefore, the name). Existing methods for time-based action localization with a few recordings use MAML frameworks or learning sequence matching networks. In both cases, a proposed action is independently compared to action examples. The inventors of the claimed innovation were the first to recognize that improved accuracy / precision in temporal action localization can be achieved with few shots by considering relationships between the action examples themselves. Various embodiments of the claimed innovation achieve this by modeling the auxiliary sentence as a graph and convoluting the graph over a trained graph convolute network.Embodiments described herein include systems, computer implemented methods, apparatus, and / or computer program products that enable temporal action location with few shots based on graph convolution networks. In one or more embodiments, an auxiliary set may include one or more single shot auxiliary videos, each corresponding to one or more temporal action classifications. The single shot auxiliary videos may be, for example, short video chips, each short video chip showing an example of a corresponding / respective temporal action classification (e.g., a first chip showing a person in progress, a second chip showing a jumping person, a third chip showing a person dropping an object, and so forth). In various cases, each snapshot auxiliary video (and thus each temporal action classification) may correspond to an example feature vector generated by a gated recurrent unit based on the snapshot auxiliary videos (e.g., a first vector representing the travel classification, a second vector representing the jump classification, a third vector representing the throw classification, etc.). According to various aspects, a graph modeling the auxiliary set may be generated. Nodes of the graph may each correspond to the temporal action classifications (e.g., a first node corresponding to the walking classification, a second node corresponding to the jumping classification, a third node corresponding to the throwing classification, and so forth). Edges of the graph may correspond to similarities between respective temporal action classifications (e.g., an edge between the run node and the jump node may correspond to a similarity between the run classification and the jump classification, an edge between the jump node and the throw node may correspond to a similarity between the jump classification and the throw classification, an edge between the run node and the throw node may correspond to a similarity between the run classification and the throw classification, and so forth). In various embodiments, the similarities may be calculated based on cosine similarities of the example feature vectors (e.g., the similarity between the run classification and the jump classification may be a cosine similarity between the run vector and the jump vector, the similarity between the jump classification and the throw classification may be a cosine similarity between the jump vector and the throw vector, the similarity between the run classification and the throw classification may be a cosine similarity between the run vector and the throw vector, etc.). In various cases, an untrimmed input video may be received and an action to be classified may be located within the untrimmed input video (e.g., the start time in the untrimmed input video and the stop time in the untrimmed input video of the action to be classified may be determined). A proposed feature vector representing the action to be classified may be generated by a gated recurrent unit based on the untrimmed input video. In various embodiments, respective concatenations may be formed by concatenating the proposed feature vector with each of the respective example feature vectors (e.g., concatenating the proposed feature vector with the run vector, concatenating the proposed feature vector with the jump vector, concatenating the proposed feature vector with the throw vector, etc.). The respective concatenations may be input to the respective nodes and the graph may be convoluted (e.g., by a trained graph convolution network) such that the nodes of the convoluted graph may output match values. The match value of each node may correspond to a match level between the temporal action classification corresponding to that node and the action to be classified (e.g., the action shown in the untrimmed input video). In various cases, the action may be determined to belong to a temporal action classification corresponding to a highest match value.Various embodiments of the claimed innovation may be used to use hardware and / or software to solve problems that are highly technical in nature (e.g. to enable temporal action localization with few recordings based on graph convolution networks), that are not abstract and that cannot be performed as a sequence of conceptual actions by a human. Moreover, some of the processes performed may be performed by a specialized computer (e.g., graph generator, cosine similarity calculator, trained graph convolution network, etc.) to perform defined tasks associated with time action localization with few shots (e.g., generating a graph modeling a subset of time action classifications, inputting concatenations of a proposed feature vector of an action to be classified with example feature vectors corresponding to the time action classifications, respectively, into the nodes of the graph, convoluting the graph to classify the action taking into account similarities between the time action classifications in the subset, and so forth). In various embodiments, the claimed innovation may be used to solve new problems arising from advances in technology, computer networks, the Internet, and the like. According to various aspects, the claimed innovation may offer technical improvements in the field of temporal action localization with few images by modelling an auxiliary set of example actions with a graph to take into account the similarities between each of the example actions. By exploiting the relationships / similarities between the example actions, various embodiments of the claimed innovation can provide a more accurate / precise temporal action localization with few exposures than conventional systems. Such a temporal action localization with few recordings based on graph convolution networks thus represents a concrete and graspable technical improvement over the prior art.FIG. 1 illustrates a block diagram of an example non-limiting system 100 that may enable low-shot temporal action localization over graph convolution networks, in accordance with one or more embodiments described herein. As shown, a few shot temporal action localization system 102 may perform temporal action localization on an input video 106 by utilizing an auxiliary video set 104.In various embodiments, the input video 106 may be an untrimmed video of any length (e.g., a commercial advertisement, an advertising video, a 15 minute Internet slide, a full length video, a sitcom episode, security camera recordings, body camera recordings, dashcam recordings, etc.). In various embodiments, the input video 106 may be prerecorded. In various other embodiments, the input video 106 may be live streamed (or near live streamed) video (e.g., live video transmission from a security camera, live video transmission from a drone or other autonomous vehicle, live video transmission from a message reporter, live video transmission from a smartphone or other mobile device, etc.). In various cases, the input video 106 may include one or more video segments with defined start and stop times within the input video 106 (e.g., segment 1 may begin at time t 0 and end at time t 1 segment 2 may begin at time t 1 and end at time t 2 end,..., segment M may begin at time t M-1 and end at time t M as shown in FIG. 1, where M may be any positive integer). In various cases, one or more of the video segments in input video 106 may correspond to an action to be classified. For example, the input video 106 may show a person, animal, and / or other entity exerting an activity (and / or inactivity) of interest, wherein the activity of interest begins in a capture time line of the input video 106 at time t x-1 and ends at time t x where x ∈[0, M]. In such a case, a segment x (not shown in FIG. 1 ) in input video 106 may be a video that shows substantially only that portion of input video 106 that shows that activity of interest. As discussed in detail below, this activity of interest may be localized / classified via graph convolution networks by one or more embodiments of the claimed innovation.In various embodiments, the auxiliary video set 104 may be a set of one or more single shot auxiliary videos (e.g., auxiliary video 1, auxiliary video 2,..., auxiliary video N) each corresponding to one or more temporal action classifications (e.g., action class 1, action class 2,..., action class N, as shown in FIG. 1, where N may be any positive integer). In various aspects, the single shot auxiliary videos may be short video chips (e.g., on the order of a few seconds in length, longer or shorter as needed / desired, etc.) that demonstrate and / or show examples of actions that correspond to the temporal action classifications, respectively. For example, if action class 1 corresponds to the activity walking, auxiliary video set 104 may include a first single shot auxiliary video (e.g., auxiliary video 1) showing an example of a person, animal, and / or other entity that is performing the activity walking (e.g., an athlete walking a marathon). Similarly, if action class 2 corresponds to jump activity, auxiliary video set 104 may include a second single shot auxiliary video (e.g., auxiliary video 2) showing an example of a person, animal, and / or other entity that is performing jump activity (e.g., a person jumping on a trampoline). Further, if an action class N corresponds to the activity of dropping an object, the auxiliary video set 104 may include an N-th single shot auxiliary video (e.g., auxiliary video N) showing an example of a person, animal, and / or other entity exerting the activity of dropping (e.g., a sporter dropping a ball). Although the activities walking, jumping, and casting are herein treated to demonstrate various aspects and embodiments of the claimed innovation, this disclosure is not limited thereby and any suitable activities and / or inactivitys that can be represented in short video chips can be included. According to various aspects, any suitable number and / or type of temporal action classifications and corresponding single-shot auxiliary videos may be included in the auxiliary video set 104 (e.g., walking, jumping, casting, playing, lifting, wearing, eating, drinking, combating, slipping, dropping, rolling, sitting, curving, climbing, driving, speaking, lading, wineing, etc.). In various aspects, any suitable / desired level of generality / specificity may be used when determining the temporal action classifications and corresponding single shot auxiliary videos in the auxiliary video set 104 (e.g., playing any competitive activity as opposed to playing any competitive activity, casting any object as opposed to casting any particular object, eating any foods as opposed to eating particular foods, operating any vehicle as opposed to operating any particular vehicle, etc.). For example, in some embodiments, a single temporal action classification may correspond to the general activity of jumping, and so each segment of input video 106 showing a person, animal, and / or other entity jumping into the air may be classified as pertaining to temporal action classification jumping, regardless of whether the person, animal, and / or other entity performed a jump-up (e.g., a light athletic activity), a jump-up (e.g., another light athletic activity), a stick jump-up (e.g., another light athletic activity), or just a jump-up (e.g., a game and / or a persistence training). In other embodiments, more specific temporal action classifications (and thus more specific single shot auxiliary videos) may be used. For example, a first temporal action classification may correspond to the activity jump, and a first single shot auxiliary video may show an example of an athlete executing a jump, a second temporal action classification may correspond to the activity jump, and a second single shot auxiliary video may show an example of an athlete executing a jump, a third temporal action classification may correspond to the activity jump, and a third single shot auxiliary video may show an example of an athlete executing a jump, a fourth temporal action classification may correspond to the activity jump, and a fourth single shot auxiliary video may show an example of a person using a jump rope, and so on. As discussed in more detail below, various embodiments of the claimed innovation may utilize relationships between each of the temporal action classifications (e.g., action class 1, action class 2,..., action class N) in the auxiliary video set 104 by manipulating vectors generated based on the corresponding single shot auxiliary videos (e.g., auxiliary video 1, auxiliary video 2,..., auxiliary video N) to classify / locate actions shown in desired video segments of the input video 106.In one or more embodiments, the claimed innovation may be generalized beyond video analysis. For example, the systems taught herein, computer implemented methods, apparatus, and / or computer program products may be used in some embodiments to locate / classify actions of interest shown in input video 106 based on the action classifications and examples included in auxiliary video set 104. However, in other embodiments, the systems taught herein, methods implemented by a computer, apparatus, and / or computer program products may be used to locate / classify sounds (e.g., using an input audio file corresponding to the input video 106 and using an auxiliary audio set corresponding to the auxiliary video set 104). In various embodiments, the systems taught herein, methods implemented by a computer, apparatus, and / or computer program products may be implemented to locate / classify any type of information in an electronic input file (e.g., input video 106), for which there are suitable examples (e.g., auxiliary video set 104). Non-limiting examples of applications may include visual classification / localization of actions, acoustic classification / localization of music and / or music patterns, acoustic classification / localization of speech, olfactory classification / localization of odors (e.g., using chemical odor sensors), tactile classification / localization (e.g., using weight distribution sensors), etc.In various embodiments, the low-shot temporal action localization system 102 may include a processor 108 (e.g., a computer processing unit, a microprocessor, etc.) and a computer readable memory 110 operatively connected to the processor 108. The memory 110 may store computer-executable instructions that, when executed by the processor 108, may cause the processor 108 and / or other components of the low-shot temporal action localization system 102 (e.g., auxiliary vector component 112, input vector component 114, graph component 116, instantiation component 118, convolution component 120, etc.) to perform one or more actions. In various embodiments, the memory 110 may store computer-executable components (e.g., auxiliary vector component 112, input vector component 114, graph component 116, instantiation component 118, convolution component 120, etc.), and the processor 108 may execute the computer-executable components.In one or more embodiments, the low-shot temporal action localization system 102 may generate, via an auxiliary vector component 112, one or more example feature vectors that respectively correspond to the one or more temporal action classifications in the auxiliary video set 104, based on the single-shot auxiliary videos in the auxiliary video set 104. In other words, the auxiliary vector component 112 may generate an action embedding in which each single-shot auxiliary video (and thus each action classification) in the auxiliary video set 104 has a corresponding vector (e.g., an example feature vector) representing it. In various embodiments, the auxiliary vector component 112 may use one or more gated recurrent units (GRUs) to generate such vectors and / or action embeds. A GRU may be a gate control mechanism used in various types of artificial neural networks to avoid the problem of the disappearing gradient during backward propagation. In various embodiments, other suitable methods for creating action embeddings (e.g., mapping visually perceived actions to vectors) may also be implemented. In various embodiments, the auxiliary vector component 112 may receive and analyze the single shot auxiliary videos (e.g., auxiliary video 1, auxiliary video 2,..., auxiliary video N) corresponding to the one or more temporal action classifications (e.g., action class 1, action class 2,..., action class N) in the auxiliary video set 104, and may generate one or more vectors (referred to as example feature vectors) corresponding to one or more temporal action classifications. For example, if action class 1 corresponds to the activity walking and auxiliary video 1 shows an example of a walking person, auxiliary vector component 112 may generate a first example feature vector (a vector having any suitable dimensionality) representing the activity walking based on auxiliary video 1. Similarly, if action class 2 corresponds to jumping activity and auxiliary video 2 shows an example of a jumping person, auxiliary vector component 112 may generate a second example feature vector representing jumping activity based on auxiliary video 2. Further, if the action class N corresponds to the activity throw and the auxiliary video N represents an example of a person casting an object, the auxiliary vector component 112 may generate an example N-th feature vector representing the activity throw based on the auxiliary video N. In various embodiments, any currently known or future developed suitable technique for mapping vectors to activities / actions, each represented in single shot auxiliary video, based on the visually perceived features / characteristics of the actions may be used (e.g., GRUs, other methods for creating action embeds, etc.).In one or more embodiments, the low-shot temporal action localization system 102 may generate, via an input vector component 114, a proposed feature vector representing an action to be classified from the input video 106. Similar to the auxiliary vector component 112, the input vector component 114 may generate a vector (e.g., an action embedding) representing an action to be classified, wherein the action to be classified is detected in at least one video segment of the input video 106. In various embodiments, the input vector component 114 may generate this vector (referred to as a proposed feature vector) via one or more GRUs and / or any other suitable method currently known or later developed. In various embodiments, the input vector component 114 may receive and analyze a video segment from the input video 106 that shows the action to be classified and output a vector (e.g., a suggested feature vector) that represents the action to be classified. For example, if a segment x (not shown in FIG. 1 ) of the input video 106 shows a person exerting an activity of interest, the input vector component 114 may receive the segment x and generate a proposed feature vector representing the activity of interest based on the visually perceived features / characteristics of the activity. As discussed in more detail below, this proposed feature vector, along with the example feature vectors generated by the auxiliary vector component 112, may be used to classify the activity of interest as one or more of the temporal action classifications in the auxiliary video set 104 (e.g., action class 1, action class 2,..., action class N).In various embodiments, the low-shot temporal action localization system 102 may generate, via a graph component 116, a graph modeling the auxiliary video set 104. In various embodiments, the graph may capture the relationships between the temporal action classifications in the auxiliary video set 104 (e.g., capture how action class 1, action class 2,..., and action class N are related to each other). In particular, nodes of the graph can correspond to the temporal action classifications. For example, the graph may have a first node corresponding to action class 1, a second node corresponding to action class 2, an Nthnode corresponding to action class N, and so forth. Moreover, edges of the graph may correspond to similarities between the temporal action classifications (and / or have corresponding weights). For example, an edge between the node corresponding to action class 1 and the node corresponding to action class 2 may represent a similarity value between action class 1 and action class 2 (e.g., a similarity between auxiliary video 1 and auxiliary video 2). According to various aspects, this similarity value may be calculated as cosine similarity between the example feature vector associated with action class 1 and the example feature vector associated with action class 2. In various embodiments, other similarity measures may also be implemented (e.g., Euclidean distance between example feature vectors, normalized Euclidean distance between example feature vectors, etc.). In various embodiments, larger similarity values may be used to identify more similar example feature vectors (e.g., the similarity value may be cos(θ), where θ represents the angle between two example feature vectors such that similarity values closer to 1 indicate greater similarity and similarity values closer to 0 indicate less similarity). In various other embodiments, smaller similarity values may be used to identify more similar example feature vectors (e.g., the similarity value may be sin(θ), where θ represents the angle between two example feature vectors, such that similarity values closer to 0 indicate greater similarity and similarity values closer to 1 indicate less similarity). In various embodiments, other methods for mathematically measuring the similarity between two example feature vectors may also be implemented. In various embodiments, the graph generated by the graph component 116 may be non-directional. According to various aspects, an undirected graph having N node edges (e.g., a full and / or full connected graph). In various embodiments, a (directed or non-directed) graph may have fewer edges. In any case, the graph component 116 may generate a graph whose nodes correspond to the temporal action classifications in the auxiliary video set 104, respectively, and whose edges correspond to similarities between the respective temporal action classifications. By modeling the auxiliary video set 104 with such a graph, the relationships between the single shot auxiliary videos (e.g., between the action examples) in the auxiliary video set 104 may be quantified and utilized to increase the effectiveness of the temporal action localization / classification. As discussed above, conventional systems that enable time-based action localization with few shots do not take these relationships into account. Instead, conventional methods compare the action to be classified independently with each example of action without considering how the examples of action are related to each other. Experimental results show that various embodiments of the claimed innovation exceed conventional systems, just because the claimed innovation uses the relationships between the temporal action classifications in the auxiliary video set 104.In various embodiments, the low-shot temporal action localization system 102 may input certain input vectors into the nodes of the graph generated by the graph component 116 via an instantiation component 118. In various cases, these input vectors may be considered initial and / or instantiated features / characteristics of the graph. In various cases, the input vectors may be based on the proposed feature vector of the action to be classified. In various cases, the input vectors may be concatenations of the proposed feature vector with corresponding example feature vectors (e.g., the input vector for the first node may be a concatenation of the proposed feature vector and the example feature vector corresponding to action class 1, the input vector for the second node may be a concatenation of the proposed feature vector and the example feature vector corresponding to action class 2, the input vector for the Nth node may be a concatenation of the proposed feature vector and the example feature vector corresponding to action class N, etc.). In various embodiments, the concatenations may be formed by associating an end of the proposed feature vector with an end of the respective example feature vector (e.g., if the proposed feature vector is a p-by-1 vector for a positive integer p and the respective example feature vector is also a p-by-1 vector, then the concatenation may be a 2p-by-1 vector). In various cases, the proposed feature vector may be listed in the concatenation in the first place. In various cases, the respective example feature vector may be listed in the concatenation in the first place. Thus, in various embodiments, the graph generated by the graph component 116 may include nodes corresponding to the respective temporal action classifications, as well as edges corresponding to similarities between the respective temporal action classifications. Moreover, the nodes may be instantiated with respective concatenations of the proposed feature vector with respective example feature vectors. In various cases, such a graph structure may be used to improve the accuracy / precision of temporal action localization with few recordings compared to conventional methods.In various embodiments, the low-shot temporal action localization system 102 may perform convolution of the graph generated by the graph component 116 via a convolution component 120 after the respective concatenations are input by the instantiation component 118. In various embodiments, convolution component 120 may employ a graph convolution network (e.g., a convolutional neural network that operates on graphs) to perform the convolution. According to various aspects, a graph convolution network is a convolution neural network (e.g., a specific type of artificial neural network) that takes as input a graph having a feature / property description for each node of the graph and also takes as input a representative description of the graph structure in matrix form (e.g., an adjacence matrix or a function thereof) and generates a node-level output (e.g., an output value for each node of the graph). In various embodiments, convolution component 120 may utilize a graph convolution network to perform convolution on the graph generated by graph component 116 and instantiated by instantiation component 118, which may cause the nodes of the convoluted graph to output match values. Each matching value may be a value indicating a degree of matching between the temporal action classification corresponding to the respective node and the action to be classified. In various embodiments, it may be determined that the action to be classified is associated with a temporal action classification having a highest match value (e.g., when numerically higher match values indicate a higher probability of match). In various embodiments, it may be determined that the action to be classified belongs to a temporal action classification with a lowest match value (e.g., if numerically lower match values indicate a higher match probability). In various embodiments, the graph convolution network used by convolution component 120 may be trained (e.g., supervised training, unsupervised training, reinforcement learning, etc.) to take as input graph structures and generate as output node-level match values. As discussed above, conventional techniques that enable temporal action localization with few shots do not model the auxiliary video set 104 as a graph and do not perform graph convolution to classify / localize actions of interest. However, the experimental results of the inventors indicate that a temporal action localization with few recordings via graph convolution networks achieves a significantly better accuracy than conventional methods for the same data sets.The following non-limiting example is provided to clarify FIG. 1. Assume that input video 106 is a commercial for fitness studio membership that includes a video segment showing a sport person climbing a rock wall. Further, it is assumed that this video segment contains the action to be classified, i.e., for the low-shot temporal action localization system 102, the ultimate goal may be to determine that the video segment in question is presenting a person performing the climbing activity. Now, assume that the auxiliary video set 104 includes three temporal action classifications (e.g., swimming, climbing, and lifting), and thus also three single shot auxiliary videos showing examples of these three temporal action classifications (e.g., a first short video tongue showing an example of a person swimming in a pool, a second short video tongue showing an example of a person climbing on a tree, and a third short video tongue showing an example of a person lifting a longhand). In this non-limiting example, the auxiliary vector component 112 may receive the auxiliary video set 104 and generate an example feature vector for each temporal action classification based on the single shot auxiliary videos (e.g., a first example feature vector representing the swimming example, a second example feature vector representing the climbing example, and a third example feature vector representing the lifting example). Similarly, the input vector component 114 may receive the video segment containing the action to be classified (e.g., the sport person climbing on the rock wall) from the input video 106 and generate, based on the video segment, a proposed feature vector representing the action to be classified (e.g., a vector representing the sport person climbing on the rock wall). According to various aspects, the graph component 116 may generate a non-directional graph modeling the auxiliary video set 104. In particular, the graph may have three nodes (e.g., a first node corresponding to the temporal action classification of swimming, a second node corresponding to the temporal action classification of climbing, and a third node corresponding to the temporal action classification of lifting). Moreover, the graph may have three edges (e.g., with N=3), where the edges represent similarities between the three temporal action classifications (e.g., a first edge between the floating node and the climbing node corresponding to the similarity between the floating classification and the climbing classification, a second edge between the climbing node and the lifting node corresponding to the similarity between the climbing classification and the lifting classification, and a third edge between the floating node and the lifting node corresponding to the similarity between the floating classification and the lifting classification). According to various aspects, the similarity values may be calculated as cosine similarities between the respective example feature vectors (e.g., the weight of the first edge is calculated as cosine similarity between the example feature vector swimming and the example feature vector climbing, the weight of the second edge is calculated as cosine similarity between the example feature vector climbing and the example feature vector lifting, and the weight of the third edge is calculated as cosine similarity between the example feature vector swimming and the example feature vector lifting). In various cases, the instantiation component 118 may input, to each node of the graph, a corresponding concatenation of the proposed feature vector and the respective example feature vector of the node (e.g., to the float node, a concatenation of the proposed feature vector and the example feature vector float, to the climb node, a concatenation of the proposed feature vector and the example feature vector climb, and to the lift node, a concatenation of the proposed feature vector and the example feature vector lift). According to various aspects, convolution component 120 may perform convolution on the generated and instantiated graph via a trained graph convolution network. In various cases, the nodes of the convoluted graph may each output a match value. In this case, the match value output from the climbing node may be higher than the match values output from the floating node or the lifting node. Thus, it may be determined that the proposed feature vector (and thus the action to be classified) is most similar to the example feature vector climbing, which indicates that the action (e.g., the sporter climbing on the rock wall) may be classified as climbing.As a simplified and non-limiting explanation, various embodiments of the claimed innovation may utilize relationships between temporal action classifications in an auxiliary set using graph convolution networks to increase the effectiveness of action localization / classification. Simply stated, conventional techniques for time action classification with few shots do not model the relationships between the time action classifications (e.g., the action classes) as graphs, nor do they use graph convolution networks to classify / locate actions.FIG. 2 illustrates a block diagram of an example, non-limiting system 200 having example feature vectors that may enable low-shot temporal action localization over graph convolution networks, in accordance with one or more embodiments described herein. As shown, in various embodiments, system 200 may include the same components as system 100, and may further include example feature vectors 202.As discussed above, in one or more embodiments, the auxiliary vector component 112 may receive the auxiliary video set 104 and generate example feature vectors 202 based on the visually perceived features / characteristics displayed in the single shot auxiliary videos (e.g., auxiliary video 1, auxiliary video 2,..., auxiliary video N) in the auxiliary video set 104. According to various aspects, the example feature vectors 202 may include a separate vector corresponding to each temporal action classification in the auxiliary video set 104 (e.g., vector 1 may be based on auxiliary video 1 and correspond to action class 1, vector 2 may be based on auxiliary video 2 and correspond to action class 2,..., vector N may be based on auxiliary video N and correspond to action class N, etc.). In various cases, each of these example feature vectors may represent its corresponding example action (e.g., vector 1 may represent the features / characteristics of the example action represented in auxiliary video 1, vector 2 may represent the features / characteristics of the example action represented in auxiliary video 2,..., vector N may represent the features / characteristics of the example action represented in auxiliary video N, etc.). As discussed above, in various embodiments of the claimed innovation, any suitable method for generating the example feature vectors 202 based on the auxiliary video set 104 can be incorporated, which is currently known or will be developed in the future. For example, the auxiliary vector component 112 may use one or more gated recurrent units to analyze each of the single shot auxiliary videos in the auxiliary video set 104 and thereby assign a unique vector to each single shot auxiliary video (and thus each temporal action classification) based on the features / characteristics represented by that single shot auxiliary video. In various cases, any other suitable action embedding method may be implemented that may map a visually perceived action to a unique numerical vector based on the features / characteristics of that visually perceived action. To continue the above example, where the input video 106 is a commercial for fitness studio membership and a video segment of the input video 106 shows a sporter climbing a rock wall, action class 1 may correspond to a swimming classification, action class 2 may correspond to a climbing classification, and action class N may correspond to a raising classification. Each of these classifications may have a respective single shot auxiliary video in the auxiliary video set 104 (e.g., the auxiliary video 1 may be a short video snippet showing an example of a floating person, the auxiliary video 2 may be a short video snippet showing an example of a climbing person, and the auxiliary video N may be a short video snippet showing an example of a person raising weights). Based on the auxiliary video 1, the auxiliary video 2, and the auxiliary video N, the auxiliary vector component 112 may generate the corresponding example feature vectors 202, where vector 1 represents the auxiliary video 1 and the action class 1, vector 2 represents the auxiliary video 2 and the action class 2, and vector N represents the auxiliary video N and the action class N.FIG. 3 illustrates a block diagram of an example, non-limiting system having a proposed feature vector that may enable temporal action localization with few recordings over graph convolution networks, in accordance with one or more embodiments described herein. As shown, in various embodiments, system 300 may include the same components as system 200 and may further include a proposed feature vector 302.As discussed above, in one or more embodiments, the input vector component 114 may receive a video segment of the input video 106 that indicates an action to classify, and may generate the proposed feature vector 302 based on the received video segment. According to various aspects, the proposed feature vector 302 may be a numerical vector representing the action to be classified and may be based on the visually perceived features / characteristics of the action to be classified represented in the received video segment. As discussed above, in various embodiments of the claimed innovation, any suitable method of generating the proposed feature vector 302 based on the received video segment from the input video 106, which is currently known or will be developed in the future, may be incorporated. For example, the input vector component 114 may use one or more gated recurrent units to analyze the action to be classified in the received video segment from the input video 106 and thereby assign a unique vector to the action to be classified that quantitates the features / characteristics of the action. In various cases, any other suitable action embedding method may be implemented that may map a visually perceived action to a unique numerical vector based on the features / characteristics of that visually perceived action. To continue the above example with the fitness studio membership commercial showing a sport person climbing a rock wall, the input vector component 114 may receive from the input video 106 a video segment showing the sport person climbing the rock wall. Based on the received video segment, the input vector component 114 may generate the proposed feature vector 302, where the proposed feature vector 302 is a numerical vector representing and / or quantifying the snippet of the sportian during climbing the rock wall. As discussed above, the proposed feature vector 302 may be used along with the example feature vectors 202 to perform a temporal action localization with few recordings over graph convolution networks.FIG. 4 illustrates a block diagram of an example non-limiting system 400 including a graph that may enable low-shot temporal action localization over graph convolution networks, in accordance with one or more embodiments described herein. As shown, in various embodiments, system 400 may include the same components as system 300 and may further include a graph 402.As discussed above, in various embodiments, the graph component 116 may generate the graph 402 modeling the auxiliary video set 104. In particular, graph 402 may model / capture the relationships between the temporal action classifications in auxiliary video set 104 (e.g., it captures how action class 1 exemplified by auxiliary video 1 is related to any other action class in auxiliary video set 104, such as action class 2 exemplified by auxiliary video 2 is related to any other action class in auxiliary video set 104,..., as action class N exemplified by auxiliary video N is related to any other action class in auxiliary video set 104, etc.). In particular, the nodes of graph 402 may each correspond to a respective temporal action classification (e.g., a first node in graph 402 may correspond to action class 1, a second node in graph 402 may correspond to action class 2,..., an Nth node in graph 402 may correspond to action class N, etc.). Additionally, the edges (e.g., and / or weights associated with the edges) of the graph 402 may correspond to similarities between respective temporal action classifications (e.g., an edge between the first node and the second node may correspond to a similarity between action class 1 and action class 2, an edge between the second node and the Nthnode may correspond to a similarity between action class 2 and action class N, a similarity between the first node and the Nthnode may correspond to a similarity between action class 1 and action class N, etc.). In various embodiments, these similarities may be calculated based on cosine similarities between respective example feature vectors 202 (e.g., the similarity between action class 1 and action class 2 may be calculated as the cosine similarity between vector 1 and vector 2, the similarity between action class 2 and action class N may be calculated as the cosine similarity between vector 2 and vector N, the similarity between action class 1 and action class N may be calculated as the cosine similarity between vector 1 and vector N, etc.). In various cases, the cosine similarity is two vectors, and simply the cosine of the angle between these vectors (e.g., cos(θ)). This can be calculated with the aid of the Euclidean point product according to the following formula:In various embodiments, other mathematical similarity measures may also be used (e.g., Euclidean distance, etc.). In various embodiments, the graph 402 may be non-directional. According to various aspects, graph 402 may have N nodes (e.g., the same number as temporal action classifications in auxiliary video set 104) and may have 1 / 2 N(N- 1) edges (e.g., a full and / or full connected graph). In various embodiments, the (directed or non-directed) graph 402 may have fewer edges (e.g., an incomplete graph). To continue the above example, where the input video 106 is a commercial for fitness studio membership, the graph 402 may include three nodes corresponding to the three temporal action classifications (e.g., a first node corresponding to the action class swimming, a second node corresponding to the action class climbing, and a third node corresponding to the action class raising). The weight of the edge between the first node and the second node may correspond to the similarity between the swimming action class and the climbing action class, the weight of the edge between the second node and the third node may correspond to the similarity between the climbing action class and the raising action class, and the weight of the edge between the first node and the third node may correspond to the similarity between the swimming action class and the raising action class.FIG. 5 illustrates a diagram 500 of an example, non-limiting graph that enables temporal action localization with few recordings over graph convolution networks, according to one or more embodiments described herein.In various embodiments, the graph 500 may be a more detailed representation of the graph 402 generated by the graph component 116. As shown, in one or more embodiments, graph 402 may include a node 1 that may correspond to action class 1 in auxiliary video set 104, a node 2 that may correspond to action class 2 in auxiliary video set 104, a node 3 that may correspond to action class 3 (not shown in FIG. 4 ) in auxiliary video set 104, and a node N that corresponds to action class N in auxiliary video set 104. Although only four nodes are shown in graph 402 in FIG. 5, any suitable / desired number of nodes may be included (e.g., N may be any suitable / desired positive integer). In some embodiments, the number of nodes of graph 402 may be equal to the number of temporal action classifications in auxiliary video set 104. As shown, an edge between node 1 and node 2 may represent a similarity 1,2 (e.g., a similarity value between action class 1 and action class 2). As discussed in detail above, the similarity value may be calculated via cosine similarity calculation between vector 1 (e.g., corresponding to action class 1 and generated based on auxiliary video 1) and vector 2 (e.g., corresponding to action class 2 and generated based on auxiliary video 2). Similarly, an edge between node 1 and node 3 may represent a similarity 1,3 calculated via a cosine similarity calculation between vector 1 and vector 3 (not shown in FIG. 4 ); an edge between node 2 and node 3 may represent a similarity 2,3 calculated via a cosine similarity calculation between vector 2 and vector 3; an edge between node 1 and node N may represent a similarity 1,N calculated via a cosine similarity calculation between vector 1 and vector N; an edge between node 2 and node N may represent a similarity 2,N calculated via a cosine similarity calculation between vector 2 and vector N; and an edge between node 3 and node N may represent a similarity 3,N calculated via a cosine similarity calculation between vector 3 and vector N. As shown, in various embodiments, graph 402 may be fully and / or fully connected, and so graph 402 may have edges when graph 402 has N nodes. In the non-limiting example illustrated in FIG. 5, graph 402 has four nodes, and thus six edges (e.g., 0.5*4*(4-1)=6). As discussed in detail above, modeling the auxiliary video set 104 via the graph 402 may capture and / or quantify relationships between the temporal action classifications in the auxiliary video set 104 (e.g., quantify relationships between action class 1, action class 2,..., and action class N). Utilizing these relationships over a graph convolution network as described herein can significantly improve the effectiveness of temporal action localization with few recordings.FIG. 6 illustrates a block diagram of an example non-limiting input vector system 600 that may enable low-shot temporal action localization over graph convolution networks, in accordance with one or more embodiments described herein. As shown, in various embodiments, system 600 may include the same components as system 400, and may further include input vectors 602.As discussed above, in various embodiments, the instantiation component 118 may generate input vectors 602 based on the proposed feature vector 302 and / or based on the example feature vectors 202. In various cases, the input vectors 602 may include concatenations corresponding to the nodes of the graph 402, respectively (e.g., N concatenations corresponding to N nodes in the graph 402, which may correspond to N temporal action classifications in the auxiliary video set 104). In particular, each vector of the input vectors 602 may be a concatenation of the proposed feature vector 302 and a corresponding vector of the example feature vectors 202 (e.g., concatenation 1 may be a concatenation of the proposed feature vector 302 with vector 1, concatenation 2 may be a concatenation of the proposed feature vector 302 with vector 2,..., concatenation N may be a concatenation of the proposed feature vector 302 with vector N, etc.). In various cases, vector chaining may be performed by linking an end of the proposed feature vector 302 to an end of a corresponding vector of the example feature vectors 202. Thus, if the proposed feature vector 302 is a p-dimensional vector for a positive integer p and the example feature vectors 202 are also p-dimensional vectors, the input vectors 602 may be 2p-dimensional vectors. In some cases, the proposed feature vector 302 may be listed first in the concatenations (e.g., the first p elements in each of the input vectors 602 may originate from the proposed feature vector 302). In some cases, the proposed feature vector 302 may be last listed in the concatenations (e.g., the last p elements in each of the input vectors 602 may originate from the proposed feature vector 302). According to various aspects, instantiation component 118 may instantiate (e.g., input) each of input vectors 602 into corresponding nodes of graph 402 (e.g., node 1 may be instantiated with concatenation 1, node 2 may be instantiated with concatenation 2,..., node N may be instantiated with concatenation N, etc.). In various aspects, the input vectors 602 may be considered initial and / or instantiated features / characteristics of the nodes of the graph 402. Continuing the above example where input video 106 is a commercial for fitness studio membership, instantiation component 118 may input concatenation 1 into the first node of graph 402, input concatenation 2 into the second node of graph 402, and input concatenation 3 into the third node of graph 402 (not shown in FIG. 6 ).FIG. 7 illustrates a diagram 700 of an example non-limiting graph with input concatenations that may enable temporal action localization with few recordings over graph convolution networks, according to one or more embodiments described herein.In various embodiments, graph 700 may be a more detailed representation of graph 402 generated by graph component 116 and instantiated by instantiation component 118. As shown, node 1 of graph 402 may receive as input (e.g., as initial / s and / or instantiated / s feature / property) concatenation 1 generated by instantiation component 118. As shown, concatenation 1 may be formed by associating proposed feature vector 302 end-to-end with vector 1 of example feature vectors 202. In the embodiment shown in FIG. 7, the proposed feature vector 302 in concatenation 1 may be listed first (e.g., when concatenation 1 includes 2pelements, the first pelements of concatenation 1 may be the elements of the proposed feature vector 302, and the last pelements of concatenation 1 may be the elements of vector 1 of the example feature vectors 202). In other embodiments, the proposed feature vector 302 may be last listed in concatenation 1 (e.g., if concatenation 1 includes 2pelements, the last pelements of concatenation 1 may be the elements of the proposed feature vector 302, and the first pelements of concatenation 1 may be the elements of vector 1 of the example feature vectors 202). In various embodiments, other ways of concatenating the proposed feature vector 302 with respective vectors of the example feature vectors 202 may also be included. As shown, as with concatenation 1, concatenation 2 may include proposed feature vector 302 and vector 2 of example feature vectors 202 and input to node 2 of graph 402, concatenation 3 (not shown in FIG. 6 ) may include proposed feature vector 302 and vector 3 (not shown in FIG. 6 ) of example feature vectors 202 and input to node 3 of graph 402, and concatenation N may include proposed feature vector 302 and vector N of example feature vectors 202 and input to node N of graph 402.FIG. 8 illustrates a block diagram of an example non-limiting system 800 with a graph convolution network that may enable low-shot temporal action localization over graph convolution networks, in accordance with one or more embodiments described herein. As shown, in various embodiments, system 800 may include the same components as system 600 and further include a GCN 802, GCN may represent graph convolutional network (graph convolution network).As discussed above, in various embodiments, convolution component 120 may perform, via GCN 802, convolution on graph 402 generated by graph component 116 and instantiated by instantiation component 118. In one or more embodiments, the GCN 802 may be a graph convolution network (e.g., a convolutional neural network operating on structured and / or unstructured graphs) that may perform one or more convolutions on the graph 402. Graph folds (e.g., spectral, spatial, etc.) are mathematical operations performed by convolutional neural networks where the convolutional neural network learns a function that takes as input a graph having a feature / property description for each node and generates a node-level output. In various embodiments, the GCN 802 may have any suitable / desired number of hidden layers and / or any suitable / desired number of neurons. In various embodiments, any suitable / desired activation function may be used by the GCN 802. In one or more cases, the GCN 802 may be trained (e.g., by supervised training, unsupervised training, reinforcement learning, etc.) to take as input the graph 402 as generated by the graph component 116 and instantiated by the instantiation component 118, and to generate a convoluted graph in which each node outputs a match value that indicates a degree of match between the action to be classified and the temporal action classification associated with the node. In various embodiments, any suitable / desired method for updating weight parameters of the GCN 802 may be implemented (e.g., backpropagation, etc.). According to various aspects, the initial feature description of each node in graph 402 may be input vectors 602 (e.g., the initial feature description of node 1 of graph 402 may be concatenation 1 of input vectors 602, the initial feature description of node 2 of graph 402 may be concatenation 2 of input vectors 602,..., the initial feature description of node N of graph 402 may be concatenation N of input vectors 602, etc.). According to various aspects, the match values may be the node-level output generated by the GCN 802. In various embodiments, it may be determined that the action to be classified belongs to a temporal action classification having a highest match value. In various embodiments, the match value for each node may be a scalar value. Thus, in various embodiments, GCN 802 may take as input graph 402, where each node has a vectorial property / feature description (e.g., one of input vectors 602), and generate as output a convoluted graph having the same number of nodes as graph 402, where each node of the convoluted graph has a scalar property / feature description (e.g., the match value). Continuing with the above example, where input video 106 is a commercial for fitness studio membership, graph 402 may be convoluted by convolution component 120 via GCN 802, and the nodes of graph 402 may output match values after convolution, with the highest match value being output by the second node of graph 402 (e.g., the node corresponding to the classification climbing). Thus, the action to be classified (e.g., the video snippet of a sport person climbing a rock wall) may be determined as an instance of a climbing action. Note that this classification / localization can be achieved without having to train a machine learning algorithm with comprehensive data showing various examples of swimming, climbing, and lifting. Instead, in various embodiments, a single example may be sufficient for each temporal action classification.In various embodiments, other methods of machine learning and / or artificial intelligence may also be used. Various embodiments of the present innovation may employ artificial intelligence (AI) to enable automation of one or more features of the present innovation. The components may use different AI-based schemes to perform various embodiments / examples disclosed herein. To enable or support the numerous determinations (e.g., determine, determine, infer, calculate, predict, predict, predict, recognize, calculate, etc.) of the present innovation, components of the present innovation may examine the entirety or subset of the data they have access to and draw conclusions or determine from a series of observations captured via events and / or data about states of the system, environment, etc. Determinations may be used to identify a particular context or action, or may generate a probability distribution over states, for example. The determinations may be probabilistic, i.e., the calculation of a probability distribution over the states of interest based on an observation of data and events. Determinations can also relate to methods by which superordinate events are composed of a set of events and / or data.Such determinations may result in new events or actions being constructed from a set of observed events and / or stored event data, regardless of whether the events are in close temporal proximity to each other and whether the events and data originate from one or more event and data sources. Components disclosed herein may include various (explicitly trained (e.g., via training data) as well as implicitly trained (e.g., via observing behaviors, preferences, historical information, receiving extrinsic information, etc.)) In addition, classification schemes and / or systems (e.g., support vector machines, neural networks, expert systems, Bayesian belief networks, fuzzy logic, data fusion machines, etc.) may be used in connection with performing automatic and / or certain actions in connection with the claimed subject matter. Thus, classification schemes and / or systems may be used to automatically learn and perform a number of functions, actions, and / or determinations.A classifier may map an input attribute vector, z=(z1, z2, z3, z4, zn), to a confidence that the input belongs to a class, as represented by f(z)=convity(class). Such classification may use probabilistic and / or statistical analysis (e.g., taking into account utility and cost in the analysis) to determine an action to be automatically performed. A support vector machine (SVM) may be an example of a classifier that may be used. The SVM operates by finding a hypersurface in the space of possible inputs, the hypersurface attempting to separate the triggering criteria from the non-triggering events. The classification is hereby clearly correct for test data which, although it may be similar to training data, are not identical to these. Other directed and non-directed model classification approaches may include naiv-Bayes, Bayes' networks, decision trees, neural networks, unsharp logic models, and / or probabilistic classification models that provide other independence patterns, any of which may be used. Classification herein also includes statistical regression used to develop priority models.FIG. 9 illustrates a diagram 900 of an example non-limiting convoluted graph that may enable temporal action localization with few recordings across graph convolution nets, according to one or more embodiments described herein.In various embodiments, graph 900 may be a more detailed representation of a convoluted graph 902 generated by GCN 802 based on graph 402. As shown, in some embodiments, the convoluted graph 902 may have the same number of nodes as the graph 402. In various embodiments, the convoluted graph 902 may have the same edges and / or the same edge weights as the graph 402. In some embodiments, the convoluted graph 902 may have different edges and / or different edge weights than the graph 402. In various embodiments, each node of the convolutional graph 902 may output a match value (e.g., a scalar) that indicates a degree of match between the action to be classified (e.g., represented by the proposed feature vector 302) and the temporal action classification corresponding to the respective node (e.g., represented by one of the example feature vectors 202). In various embodiments, the match value of each node may be considered the description of the properties / features of each node of the graph after convolution.FIG. 10 illustrates a high-level block diagram of an example non-limiting system 1000 that may enable temporal action localization with few recordings over graph convolution networks, in accordance with one or more embodiments described herein.In various aspects, FIG. 10 may illustrate a non-limiting and exemplary summary of various embodiments of the claimed innovation. As shown and described above, input video 106 may show an action to classify. The auxiliary video set 104 may include one or more single shot auxiliary videos (e.g., auxiliary video 1, auxiliary video 2,..., auxiliary video N) showing examples of the respective temporal action classifications (e.g., action class 1, action class 2,..., action class N). Vectors may be created to represent the temporal action classifications (e.g., example feature vectors 202). In addition, a vector representing the action to be classified (e.g., the proposed feature vector 302) may be generated. Based on these vectors representing the auxiliary video set 104 and the input video 106, a concatenation may be generated for each of the temporal action classifications (e.g., concatenation 1, concatenation 2,..., concatenation N). As discussed above, graph 402 may be generated to model auxiliary video set 104, with a separate node for each temporal action classification in auxiliary video set 104, where the edges of graph 402 represent similarities between respective temporal action classifications. Respective nodes of graph 402 may be instantiated with the respective concatenations. The graph 402 may then be input to the GCN 802 (e.g., a convolutional neural network operating on graphs) to perform a convolution on the graph 402, thereby creating the convoluted graph 902. Each node of the convolutional graph 902 may have a scalar feature / characteristic referred to as a match value (e.g., match value 1, match value 2,..., match value N). In various cases, it may be determined that the action to be classified is associated with a temporal action classification corresponding to a node having a highest match value.FIG. 11 illustrates a table 1100 of example, non-limiting results of a few shot temporal action localization over graph convolution networks according to one or more embodiments described herein.Table 1100 shows, for a particular dataset, the mean average precision (mAP) of an embodiment of the claimed innovation (e.g., action localization through a graph convolution network) as compared to the mean average precision of a prior art method (e.g., action localization through learning sequence matching). As shown, the prior art method performed a temporal action localization with few shots on a particular dataset with an average accuracy of 13.6%. (See Yang, Hongtao, Xuming He, and Fatih Porikli: "One-Shot Action Localization by Learning Sequence Matching Network" (Single Shot Action Localization by Learning Sequence Matching Network), Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018). In contrast, an embodiment of the claimed innovation performed a temporal action localization with few recordings on the same dataset with an average average accuracy of 15.3%. This represents a considerable improvement in the effectiveness of the time-based action localization with a few recordings. As discussed in detail above, various embodiments of the claimed innovation may significantly exceed conventional methods by quantifying and utilizing (e.g., via graph 402 and convolution component 120) the relationships between the temporal action classifications in the auxiliary sentence. Conventional methods simply do not take into account how the action examples relate to each other, instead they only compare a proposed feature independently with each action example. The inventors of the claimed innovation were the first to appreciate that considerable improvements in accuracy / precision can be achieved by exploiting the relationships between the action classifications. Various embodiments of the claimed innovation achieve this by modeling the auxiliary set over a particularly structured graph (e.g., graph 402) and performing a convolution on the graph.FIG. 12 illustrates a flow diagram of an example, non-limiting, computer-implemented method 1200 that may enable low-shot temporal action localization over graph convolution networks, according to one or more embodiments described herein.In various aspects, step 1202 may include generating, by a unit operatively connected to a processor, a graph (e.g., graph 402) modelling an auxiliary set (e.g., auxiliary video set 104) of temporal action classifications (e.g., action class 1, action class 2,..., action class N). In various embodiments, the nodes (e.g., node 1, node 2,..., node N) of the graph may correspond to respective temporal action classifications in the subset (e.g., node 1 may correspond to action class 1, node 2 may correspond to action class 2,..., node N may correspond to action class N). In various embodiments, edges of the graph may correspond to similarities between the respective temporal action classifications (e.g., similarity 1,2 between node 1 and node 2 may represent similarity between action class 1 and action class 2, similarity 2,N between node 2 and node N may represent similarity between action class 2 and action class N, etc.).In various embodiments, step 1204 may include, by the entity, performing a convolution (e.g., via convolution component 120 and / or GCN 802) on the graph such that the nodes (e.g., nodes of convolutional graph 902) output respective match values (e.g., match value 1, match value 2,..., match value N) that indicate match degrees between the respective temporal action classifications and an action to be classified (e.g., match value 1 indicates a match degree between the action to be classified and action class 1, match value 2 indicates a match degree between the action to be classified and action class 2,..., Match value N shows a degree of match between the action to be classified and the action class Nan).FIG. 13 illustrates a flow diagram of an example, non-limiting, computer-implemented method 1300 of input vectors that may enable temporal action localization with few recordings over graph convolution networks, in accordance with one or more embodiments described herein. As shown, in various embodiments, the computer implemented method 1300 may include the same steps as the computer implemented method 1200, and further include step 1302.In various cases, step 1302 may include, by the entity, inputting (e.g., via instantiation component 118) respective input vectors (e.g., input vectors 602) to the nodes based on a proposed feature vector (e.g., proposed feature vector 302) representing the action to be classified.FIG. 14 illustrates a flow diagram of an example, non-limiting, computer-implemented method 1400 with input concatenations that may enable low-shot temporal action localization over graph convolution networks, according to one or more embodiments described herein. As shown, in various embodiments, the computer implemented method 1400 may include the same steps as the computer implemented method 1300, and further include steps 1402 and 1404.According to various aspects, step 1402 may include the respective temporal action classifications corresponding to respective example feature vectors (e.g., action class 1 may correspond to vector 1 of example feature vectors 202, action class 2 may correspond to vector 2 of example feature vectors 202,..., action class N may correspond to vector N of example feature vectors 202).In various embodiments, step 1404 may include the respective input vectors being concatenations of the respective example feature vectors and the proposed feature vector representing the action to be classified (e.g., concatenation 1 may be a concatenation of the proposed feature vector 302 and the vector 1 of the example feature vectors 202, concatenation 2 may be a concatenation of the proposed feature vector 302 and the vector 2 of the example feature vectors 202,..., concatenation N may be a concatenation of the proposed feature vector 302 and the vector N of the example feature vectors 202).FIG. 15 illustrates a flow diagram of an example, non-limiting, computer-implemented gated recurrent unit method 1500 that may enable low-shot temporal action localization over graph convolution networks, according to one or more embodiments described herein. As shown, in various embodiments, the computer implemented method 1500 may include the same steps as the computer implemented method 1400, and further include steps 1502 and 1504.In various cases, step 1502 may include generating, by the unit, the respective example feature vectors via a first gated recurrent unit (e.g., used by the auxiliary vector component 112) based on respective single shot auxiliary videos (e.g., auxiliary video 1, auxiliary video 2,..., auxiliary video N) displaying examples of the respective temporal action classifications.In various aspects, step 1504 may include generating, by the unit, the proposed feature vector via a second gated recurrent unit (e.g., used by the input vector component 114) based on an untrimmed input video (e.g., input video 106) showing the action to be classified.To provide additional context for various embodiments described herein, FIG. 16 and the following discussion are intended to provide a general description of a suitable computing environment 1600 in which the various embodiments of the embodiment described herein may be implemented. While the embodiments have been described above in the general context of computer executable instructions that may run on one or more computers, one skilled in the art will recognize that the embodiments may also be implemented in combination with other program modules and / or as a combination of hardware and software.Generally, program modules include routines, programs, components, data structures, etc., that perform certain tasks or implement certain abstract data types. Moreover, one skilled in the art will understand that the methods of the present invention may be performed with other computer system configurations, including single processor or multiprocessor computer systems, minicomputers, mainframe computers, IoT (Internet of Things) devices, distributed computing systems, as well as personal computers, portable computing devices, microprocessor-based or programmable consumer electronics, and the like, each of which may be operatively connected to one or more associated devices.The illustrated embodiments of the present embodiments may also be implemented in distributed computing environments where certain tasks are performed by remote processing units interconnected by a communications network. In a distributed computing environment, program modules may reside in both local and remote memory storage devices.Computing devices typically include a variety of media, which may include computer readable storage media, machine readable storage media, and / or communication media, and these two terms are distinguished herein as follows. Computer or machine readable storage media may be any available storage media accessible by the computer, and includes both volatile and non-volatile media, removable media, and non-removable media. By way of example, and not limitation, computer- or machine-readable storage media may be implemented in connection with any method or technology for storing information, such as computer- or machine-readable instructions, program modules, structured data, or unstructured data.Computer readable storage media includes, but is not limited to, random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disk read only memory (CD-ROM), digital versatile disk (DVD), blu-ray disk (BD), or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, semiconductor storage or other semiconductor storage devices, or other tangible and / or non-transitory media that can be used to store desired information. In this regard, the terms "tangible" or "non-transient" as applied herein to storage, storage, or computer readable media are understood to exclude, as modifiers, only transient signal propagation per se, and not to give rights to any standard storage, storage, or computer readable media that not only transient signals per se.Computer readable storage media may be accessed by one or more local or remote computing devices, such as via access requests, queries, or other data retrieval protocols, to perform a variety of operations with respect to the information stored on the medium.Communication media typically includes computer readable instructions, data structures, program modules, or other structured or unstructured data in a data signal such as a modulated data signal, e.g., a carrier wave or other transport mechanism, and includes any information delivery or transport media. The term "modulated data signal" or "signals" refers to a signal in which one or more of its features have been adjusted or altered to encode information in one or more signals. Data transmission media includes, for example and not limited to, wired media such as a wired network or a direct cable connection, and wireless media such as acoustic, RF, infrared, and other wireless media.Referring again to FIG. 16, to implement various embodiments of the aspects described herein, the example environment 1600 includes a computer 1602, the computer 1602 including a processing unit 1604, a system memory 1606, and a system bus 1608. The system bus 1608 couples system components including, but not limited to, the system memory 1606 to the processing unit 1604. Processing unit 1604 may be any of a variety of commercially available processors. Dual microprocessors and other multiprocessor architectures may also be used as the processing unit 1604.The system bus 1608 may be any of several types of bus structures, which may further be connected to a memory bus (with or without a memory controller), a peripheral bus, and a local bus using any of several commercially available bus architectures. System memory 1606 includes ROM 1610 and RAM 1612. A basic input / output system (BIOS) may be stored in a non-volatile memory such as ROM, EPROM (Erasable Programmable Read Only Memory), or EEPROM, where the BIOS contains the basic routines that aid in the transfer of information between elements within the computer 1602, e.g., upon power-up. The RAM 1612 may also include high speed RAM, such as static RAM, for buffering data.The computer 1602 further includes an internal hard disk drive (HDD) 1614 (e.g., EIDE, SATA), one or more external storage units 1616 (e.g., a magnetic disk drive (FDD) 1616, a memory stick or flash drive reader, a memory card reader, etc.), and a drive 1620, e.g., a semiconductor memory, an optical disk drive capable of reading or writing from a disk 1622, e.g., a CD-ROM, a DVD, a BD, etc. If it is a semiconductor memory, the disk 1622 is not included unless it is separate. Although internal hard disk 1614 is depicted in computer 1602, it may also be configured for external use in a suitable housing (not shown). Additionally, although not shown in environment 1600, semiconductor memory (SSD) could be used in addition to or in place of HDD 1614. Hard disk 1614, external storage device(s) 1616, and drive 1620 may be connected to system bus 1608 via HDD interface 1624, external storage interface 1626, and drive interface 1628, respectively. External drive implementation interface 1624 may include at least one or both of Universal Serial Bus (USB) and Institute of Electrical and Electronics Engineers (IEEE) 1394 interface technologies. Other technologies for connecting to external drives are conceivable within the scope of the embodiments described here.The drives and associated computer readable storage media enable non-transitory storage of data, data structures, computer executable instructions, etc. For the computer 1602, the drives and storage media enable storage of any data in a suitable digital format. Although the above description of computer readable storage media refers to respective types of storage devices, it should be understood by those skilled in the art that other types of storage media that can be read by a computer, whether they already exist or are developed only in the future, can be used in the example operating environment, and that each of these storage media can include computer executable instructions for performing the methods described herein.The drives and RAM 1612 may store a number of program modules, including an operating system 1630, one or more application programs 1632, other program modules 1634, and program data 1636. The operating system, applications, modules, and / or data may also be stored wholly or partially in RAM 1612. The systems and methods described herein may be implemented using various commercially available operating systems or combinations of operating systems.Computer 1602 may optionally also include emulation technologies. For example, a hypervisor (not shown) or other intermediary may emulate a hardware environment for operating system 1630, and the emulated hardware may optionally be different from the hardware shown in FIG. 16. In such an embodiment, operating system 1630 may include a virtual machine (VM) of multiple VMs hosted on computer 1602. Moreover, operating system 1630 may provide runtime environments, such as the Java runtime environment or the.NET framework, to applications 1632. Runtime environments are consistent execution environments that allow applications 1632 to execute on any operating system that includes the runtime environment. Similarly, operating system 1630 may support containers, and applications 1632 may be in the form of containers, which are lightweight, stand-alone executable software packages containing, e.g., code, runtime, system tools, system libraries, and settings for an application.Further, the computer 1602 may be equipped with a security module, e.g., a trusted processing module (TPM). For example, in a TPM, hash boot components next hash subsequent boot components and wait for results to match the saved values before loading the next boot component. This process may occur at any level of the code execution stack of the computer 1602, e.g., at the application execution level or at the operating system kernel level, thereby enabling security at any level of code execution.A user may input commands and information to the computer 1602 via one or more wired / wireless input devices, e.g., a keyboard 1638, a touch screen 1640, and a pointing device such as a mouse 1642. Other input units (not shown) may include a microphone, an infrared (IR) remote, a radio frequency (RF) remote or other remote, a joystick, a virtual reality controller and / or headset, a gamepad, a stylus, an image input unit, e.g., one or more cameras, a gesture sensor input unit, a vision motion sensor input unit, an emotion or face recognition unit, a biometric input unit, e.g., a fingerprint or iris scanner, or the like. These and other input devices are often connected to the processing device 1604 through an input device interface 1644, which may be connected to the system bus 1608, but may also be connected through other interfaces such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, a BLUETOOTH ®- interface, etc.A monitor 1646 or other type of display unit may also be connected to the system bus 1608 via an interface, such as a video adapter 1648. In addition to the monitor 1646, a computer typically includes other peripheral output devices (not shown) such as speakers, printers, etc.The computer 1602 may operate in a networked environment via logical connections via wired and / or wireless communication with one or more remote computers, such as one or more remote computers 1650. The remote / n computer 1650 may / may be a workstation, a server computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other common network node, and typically includes / includes many or all of the elements described with respect to the computer 1602, although only one storage / storage device 1652 is shown for brevity. The illustrated logical connections include wired / wireless connectivity to a local area network (LAN) 1654, and / or larger networks, e.g., a wide area network (WAN) 1656. Such LAN and WAN networking environments are common in offices and companies, and enable corporate wide computer networks, such as intranets, all of which may be connected to a global communications network, e.g., the Internet.When used in a LAN networking environment, the computer 1602 may be connected to the local area network 1654 via a wired and / or wireless communications network interface or adapter 1658. The adapter 1658 may enable wired or wireless communication with the LAN 1654, which may also include a wireless access point (AP) disposed thereon for communicating with the adapter 1658 in a wireless mode.When deployed in a WAN networking environment, the computer 1602 may include a modem 1660 or may be connected to a communication server in the WAN 1656 via other means for establishing communication over the WAN 1656, e.g., via the Internet. The modem 1660 which may be an internal or external, wired, or wireless device may be connected to the system bus 1608 via the input device interface 1644. In a networked environment, program modules depicted with respect to the computer 1602 or portions thereof may be stored in the remote storage / storage unit 1652. It will be understood that the network connections shown are exemplary and other means of establishing a communications link between the computers may also be used.When used in a LAN or WAN networking environment, computer 1602 may access cloud storage systems or other network-based storage systems in addition to or in place of external storage devices 1616 described above, such as, but not limited to, a virtual network machine that provides one or more aspects of storing or processing information. In principle, a connection may be established between the computer 1602 and a cloud storage system via a LAN 1654 or WAN 1656, e.g., through the adapter 1658 or the modem 1660 respectively. After the computer 1602 is connected to an associated cloud storage system, the external storage interface 1626 may manage storage provided by the cloud storage system, such as other types of external storage, using the adapter 1658 and / or the modem 1660. For example, external storage interface 1626 may be configured to provide access to cloud storage sources as though these sources were physically connected to computer 1602.The computer 1602 may communicate with any wireless devices or entities wirelessly operatively connected, e.g., a printer, scanner, desktop and / or portable computer, portable data assistant, communication satellites, any devices or locations connected to a wirelessly recognizable identifier (e.g., kiss, newspaper kiss, shop shelf, etc.), and telephone. This may include Wi-Fi (wireless fidelity) and BLUETOOTH ®- wireless technologies. Thus, the data exchange may be a predefined structure as in a conventional network or simply an ad hoc data exchange between at least two entities.The present invention can be a system, a method, an apparatus and / or a computer program product of any possible technically detailed degree of integration. The computer program product may comprise computer readable storage medium(s) on / on which computer readable program instructions are / are stored for causing a processor to carry out aspects of the present invention. The computer readable storage medium may be a physical device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage unit, a magnetic storage unit, an optical storage unit, an electromagnetic storage unit, a semiconductor storage unit, or any suitable combination thereof. A non-exhaustive list of more specific examples of the computer readable storage medium may also include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions stored thereon, and any suitable combination thereof. A computer readable storage medium, as used herein, is not intended to be transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through an optical fiber cable), or electrical signals transmitted through a wire.Computer readable program instructions described herein may be downloaded from a computer readable storage medium to respective data processing / processing devices or via a network such as the Internet, a local area network, a wide area network, and / or a wireless network to an external computer or storage device. The network may include copper transmission cables, lightwave transmission conductors, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing unit receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the corresponding computing / processing unit. Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, integrated circuit configuration data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++, or the like, as well as conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter case, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, via the Internet using an Internet Service Provider). In some embodiments, electronic circuits, including, for example, programmable logic circuits, field programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuits to perform aspects of the present invention.Aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be appreciated that each block of the flowcharts and / or the block diagrams, as well as combinations of blocks in the flowcharts and / or the block diagrams, may be executed by computer readable program instructions. These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions executed via the processor of the computer or other programmable data processing apparatus produce a means for implementing the functions / steps specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions may also be stored on a computer readable storage medium that can control a computer, programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored thereon comprises an article of manufacture including instructions that implement aspects of the function / step specified in the flowchart and / or block diagrams block or blocks. The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of process steps to be performed on the computer or other programmable apparatus or other device to produce a computer executed process, such that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / steps specified in the flowchart and / or block diagram block or blocks.The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions that include one or more executable instructions for executing the particular logical function(s). In some alternative implementations, the functions indicated in the blocks may occur in a different order than shown in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order depending on the appropriate functionality. It is further noted that each block of the block diagrams and / or the flowcharts, as well as combinations of blocks in the block diagrams and / or the flowcharts, may be implemented by special purpose hardware-based systems that perform the specified functions or steps, or execute combinations of special purpose hardware and computer instructions.While the subject matter has been described above in the general context of computer executable instructions of a computer program product executing on a computer and / or computers, those skilled in the art will appreciate that this disclosure may be practiced in combination with other program modules. Generally, program modules include routines, programs, components, data structures, etc., that perform certain tasks and / or implement certain abstract data types. Moreover, one skilled in the art will appreciate that the computer implemented methods of the present invention may also be implemented with other computer system configurations, including single processor or multiprocessor computer systems, mini-computing devices, mainframe computers, as well as computers, hand-held computing devices (e.g., PDA, phone), microprocessor-based or programmable user or industrial electronics, and the like. The illustrated aspects may also be implemented in distributed computing environments where tasks are performed by remote processing units connected by a communication network. However, some, if not all, aspects of this disclosure may also be implemented on stand-alone computers. In a distributed computing environment, program modules may reside in both local and remote memory storage devices.The terms "component", "system", "platform", "interface", and the like as used in this application may refer to or include a computer-related entity or entity relating to an operable machine having one or more specific functionalities. The units disclosed herein may be either hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable, an execution thread, a program, and / or a computer. For the sake of clarity, both an application running on a server and the server can be a component. One or more components may be part of a process and / or execution thread, and one component may be located on one computer and / or distributed among two or more computers. In another example, respective components may be executed from different computer readable media having different data structures stored thereon. The components may communicate via local and / or remote processes, for example according to a signal with one or more data packets (e.g., data from a component that communicates via the signal with another component in a local system, a distributed system, and / or across a network, for example the Internet, with other systems). As another example, a component may be a device with certain functionality provided by mechanical parts operated by electrical or electronic circuitry and operated by a software application or firmware application executed by a processor. In such a case, the processor may be internal or external to the device and execute at least a portion of the software or firmware application. As another example, a component may be a device that provides certain functionality through electronic components without mechanical parts, where the electronic components may include a processor or other means to execute software or firmware that at least partially provides the functionality of the electronic components. In one aspect, a component may emulate an electronic component via a virtual machine, e.g., in a cloud computing system.Moreover, the term "or" is intended to be inclusive of "or" rather than exclusive of "or.". That is, unless otherwise stated or without context, "X applies A or B" is to be understood as any of the natural inclusive permutations. That is, if X applies A, X applies B, or X applies both A and B, then "X applies A or B" is satisfied in each of the cases mentioned. Moreover, the articles "a / e" as used in this specification and the accompanying drawings are to be broadly construed as "one / e or more" unless otherwise stated or without context requiring a singular form. The terms "example" and / or "exemplary" as used herein are used to mean "serving as an example, example case, or illustration.". For clarity, it is noted that the presently disclosed subject matter is not limited by such examples. Moreover, none of the aspects or configurations described herein as "example" and / or "exemplary" are necessarily to be construed as being preferred or advantageous over other aspects or configurations, nor are equivalent exemplary structures and methods known to those skilled in the art to be excluded thereby.The term "processor" as used in this specification may refer to substantially any computing unit or device, including, but not limited to, single core processors, single processors capable of multi-threaded execution of software, multi-core processors, multi-core processors capable of multi-threaded execution of software, multi-core processors capable of hardware multi-threaded technology, parallel platforms, and parallel platforms with distributed shared memory. In addition, a processor may refer to an integrated circuit, an application specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic controller (PLC), a complex programmable logic unit (CPLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. Further, processors may utilize nano-architectures such as, but not limited to, molecular and quantum dot based transistors, switching units, and gates to optimize space utilization or enhance user device performance. A processor can also be implemented as a combination of data processing units or processing units. In this disclosure, terms such as "memory," "storage," "data storage," "data storage," "database," and substantially any other information storage component relevant to the operation and functionality of a component are used to refer to "main storage components," units contained in a "main storage," or components comprising main storage. It should be noted that main memories and / or main memory components described herein may be, or may include both volatile main memory or nonvolatile main memory. By way of illustration and not limitation, non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), flash memory, or non-volatile random access memory (RAM) (e.g., ferroelectric RAM (FeRAM). Volatile memory may include RAM, which may function as an external cache, for example. By way of illustration and not limitation, RAM is available in many forms, such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), DOUBLE DATA RATE SDRAM (DDR SDRAM), advanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), direct rambus RAM (DRRAM), direct rambus dynamic RAM (DRDRAM), and rambus dynamic RAM (RDRAM). Moreover, the disclosed memory components of present systems or computer implemented methods are intended to include, but are not limited to, these and other suitable types of memory.The above includes merely examples of systems and computer implemented methods. Of course, it is not possible to describe any conceivable combination of components and computer implemented methods for purposes of describing this disclosure, but one skilled in the art will recognize that many other combinations and permutations of this disclosure are possible. Furthermore, the terms "comprises / includes", "has", "has", and the like, as used in the detailed description, claims, appendices, and drawings, are to be understood as being inclusive in a similar manner in which the term "comprising / having" is interpreted when used as a keyword in a claim.The descriptions of the various embodiments have been presented for purposes of illustration, but are not to be construed as exhaustive or limited to the embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application, or technical improvement over the technologies available on the market, or to enable one of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

A system (102) comprising: a memory (110) storing computer-executable components; and a processor (108) operatively coupled to the memory and executing the computer-executable components stored in the memory, the computer-executable components comprising: a graph component (116) generating a graph (402) modeling an auxiliary set (104) of temporal action classifications such that nodes of the graph correspond to respective temporal action classifications in the auxiliary set and edges of the graph correspond to similarities between the respective temporal action classifications; and a convolution component (120) performing convolution on the graph such that the nodes output respective match values indicative of degrees of match between the respective temporal action classifications and an action to be classified, the system further comprising an instantiation component (118) that inputs respective input vectors (602) to the nodes based on a proposed feature vector (302) representing the action to be classified, the respective temporal action classifications corresponding to respective example feature vectors (202); and the respective input vectors are concatenations of the respective example feature vectors and the proposed feature vector representing the action to be classified, wherein the action to be classified is an action displayed in an untrimmed input video, wherein it is determined that the action to be classified belongs to a temporal action classification having a highest match value.The system of claim 1, wherein: the respective example feature vectors are generated via a first gated recurrent unit based on respective single shot auxiliary videos showing examples of the respective temporal action classifications; and the proposed feature vector is generated via a second gated recurrent unit based on an untrimmed input video showing the action to be classified.The system of claim 1, wherein the similarities are calculated based on cosine similarities between respective example feature vectors of respective temporal action classifications.The system of claim 1, wherein the graph is a non-directional graph.A computer implemented method (1200) comprising: generating (1202), by a unit operatively connected to a processor (108), a graph (402) modeling an auxiliary set of temporal action classifications such that nodes of the graph correspond to respective temporal action classifications in the auxiliary set and edges of the graph correspond to similarities between the respective temporal action classifications; performing (1204), by the unit, a convolution on the graph such that the nodes output respective match values indicative of degrees of match between the respective temporal action classifications and an action to be classified; inputting (1302), by the unit, respective input vectors (602) to the nodes based on a proposed feature vector (302) representing the action to be classified, wherein the respective temporal action classifications correspond to respective example feature vectors (202) (1402); and the respective input vectors are concatenations of the respective example feature vectors and the proposed feature vector representing the action to be classified (1404), wherein the action to be classified is an action indicated in an untrimmed input video, wherein it is determined that the action to be classified belongs to a temporal action classification having a highest match value.The computer implemented method (1200, 1400, 1500) of claim 5, further comprising: generating (1502), by the unit, the respective example feature vectors via a first gated recurrent unit based on respective single shot auxiliary videos showing examples of the respective temporal action classifications; and generating (1504), by the unit, the proposed feature vector via a second gated recurrent unit based on an untrimmed input video showing the action to be classified.The computer implemented method of claim 5, wherein the similarities are calculated based on cosine similarities between respective example feature vectors of respective temporal action classifications.The computer implemented method of claim 5, wherein the graph is a non-directional graph.A computer program product for enabling action localization with few shots, the computer program product comprising a computer readable storage medium having program instructions thereon, the program instructions executable by a processing component to cause the processing component to: generate a graph (402) modeling an auxiliary set of temporal action classifications such that nodes of the graph correspond to respective temporal action classifications in the auxiliary set and edges of the graph correspond to similarities between the respective temporal action classifications; perform a convolution on the graph such that the nodes output respective match values indicative of degrees of match between the respective temporal action classifications and an action to be classified; based on a proposed feature vector (302) representing the action to be classified, inputting respective input vectors (602) to the nodes, the respective temporal action classifications corresponding to respective example feature vectors (202); and the respective input vectors are concatenations of the respective example feature vectors and the proposed feature vector representing the action to be classified, the action to be classified being an action displayed in an untrimmed input video, wherein it is determined that the action to be classified belongs to a temporal action classification having a highest match value.The computer program product of claim 9, wherein the program instructions are further executable to cause the processing component to: generate the respective example feature vectors via a first gated recurrent unit based on respective single shot auxiliary videos showing examples of the respective temporal action classifications; and generate the proposed feature vector via a second gated recurrent unit based on an untrimmed input video showing the action to be classified.The computer program product of claim 9, wherein the similarities are calculated based on cosine similarities between respective example feature vectors of respective temporal action classifications.

Images