An in-memory computing implementation method and device for few-shot class learning of a graph sample and electronic equipment

Abstract

This invention discloses an in-memory computation implementation method, apparatus, and electronic device for graph few-shot class augmentation learning, relating to the fields of machine learning and artificial intelligence. The in-memory computation implementation method for graph few-shot class augmentation learning includes: dividing a graph dataset into multiple rounds of class augmentation task sets according to categories, wherein each round adds a predetermined number of categories compared to the previous round; dividing each class augmentation task set into a training set and a test set; in a basic stage, training an initial binary graph few-shot class augmentation model based on the training set corresponding to the class augmentation task set in round 0; in the class augmentation stage, training a binary graph few-shot class augmentation model based on the multiple rounds of class augmentation task sets combined with the initial binary graph few-shot class augmentation model; and in the inference stage, determining the predicted category of the test sample based on the test set using the binary graph few-shot class augmentation model.

Description

Technical Field

[0001] This invention relates to the fields of machine learning and artificial intelligence, and in particular to an in-memory computation method, apparatus and electronic device for graph few-shot class augmentation learning. Background Technology

[0002] Graph-structured data is ubiquitous in the real world and has a wide range of applications, such as social networks, recommender systems, transaction recognition, and traffic prediction. Real-world graph data evolves rapidly over time, gradually introducing new nodes and edges, thus generating new categories. These new categories typically have only a small number of labels, and using a limited number of labeled samples to learn new categories can lead to catastrophic forgetting and overfitting problems.

[0003] To address the aforementioned issues, various deep learning models have been developed for graph few-shot class-incremental learning (GFSCIL). However, implementing GFSCIL on traditional digital hardware involves significant data transfer between computation and storage units, resulting in additional energy and time overhead. This makes it difficult to apply deep learning models for graph few-shot class-incremental learning in resource-constrained edge AI systems.

[0004] The above problems can be solved by using a Computing In Memory (CIM) accelerator based on a novel non-volatile memory (e.g., RRAM). However, due to the non-ideal characteristics of the device, such as conductance updates, changes, and low precision, implementing GFSCIL with a high-precision neural network model is not conducive to device programming. Furthermore, when learning new classes, the training process involves multiple weight iterations and updates. Implementing GFSCIL with a CIM accelerator requires frequent programming of the device, which not only leads to a large amount of programming energy consumption but also greatly reduces the device's durability. Summary of the Invention

[0005] The purpose of this invention is to provide an in-memory computation method, apparatus, and electronic device for graph few-shot class augmentation learning. Through software and hardware co-design, it addresses the problem of decreased recognition rate in few-shot class augmentation scenarios and the issue of frequent high-precision programming encountered when implementing GFSCIL using existing CIM methods. Existing methods for implementing GFSCIL using high-precision neural network models are not conducive to device programming. The frequent programming of devices is required during the implementation of GFSCIL using a CIM accelerator, which not only leads to significant programming energy consumption but also greatly reduces device durability.

[0006] In a first aspect, the present invention provides an in-memory computation implementation method for graph few-sample class augmentation learning, comprising:

[0007] The graph dataset is divided into multiple rounds of class-adding task sets according to categories, where each round adds a preset number of categories compared to the previous round;

[0008] Each of the aforementioned class-specific task sets is divided into a training set and a test set;

[0009] In the basic stage, an initial binary graph small sample class-increasing model is trained based on the training set corresponding to the class-increasing task set described in round 0.

[0010] In the class augmentation phase, a binary graph small sample class augmentation model is trained based on the class augmentation task set from multiple rounds and the initial binary graph small sample class augmentation model.

[0011] During the inference phase, the predicted category of the test sample is determined based on the test set using the binary graph few-sample class augmentation model.

[0012] With the above technical solution, the in-memory computation implementation method for graph few-shot class augmentation learning provided in this embodiment of the invention divides the graph dataset into multiple rounds of class augmentation task sets according to categories, wherein each round adds a preset number of categories compared to the previous round; each class augmentation task set is divided into a training set and a test set; in the basic stage, an initial binary graph few-shot class augmentation model is trained based on the training set corresponding to the class augmentation task set in round 0; in the class augmentation stage, a binary graph few-shot class augmentation model is trained based on the class augmentation task sets in multiple rounds combined with the initial binary graph few-shot class augmentation model; in the inference stage, the predicted category of the test sample is determined based on the test set using the binary graph few-shot class augmentation model, which can achieve high end-to-end node classification accuracy and effectively accelerate the convergence of the training process, reducing the number of programming operations.

[0013] In one possible implementation, the binary graph few-shot class-increasing model includes interconnected controllers and external memory units. The initial binary graph few-shot class-increasing model is trained in the basic phase based on the training set corresponding to the class-increasing task set in round 0, including:

[0014] In the basic phase, the error between the predicted class and the true class is determined based on the training set corresponding to the class augmentation task set described in round 0;

[0015] Based on the error, the parameters in the controller are updated to determine the initial binary image small sample class augmentation model that has been trained.

[0016] In one possible implementation, during the class augmentation phase, a binary graph few-shot class augmentation model is trained based on the class augmentation task set from multiple rounds combined with the initial binary graph few-shot class augmentation model, including:

[0017] During the class augmentation phase, the cross-entropy loss and distillation loss corresponding to the initial binary graph small sample class augmentation model are determined based on the class augmentation task set of multiple rounds.

[0018] Based on the cross-entropy loss and the distillation loss, update the control parameters of the binary graph small sample incremental model obtained by copying the initial binary graph small sample incremental model, and determine the trained binary graph small sample incremental model.

[0019] In one possible implementation, determining the error between the predicted class and the true class based on the training set corresponding to the class-addition task set in round 0 during the basic phase includes:

[0020] In the basic stage, a preset number of categories are randomly selected from the initial training set corresponding to the class-addition task set corresponding to the 0th round, and a preset number of samples are randomly selected from each category to form a support set and a query set;

[0021] The first binary feature vector is obtained from the samples in the support set through the controller;

[0022] The first binary feature vector and the tag corresponding to the external memory unit are stored in the external memory unit;

[0023] Determine the similarity between the second binary feature vector obtained by the controller and the first binary feature vector of the samples in the query set;

[0024] The predicted category of the query set is determined based on the similarity.

[0025] Determine the error between the predicted category and the true category.

[0026] In one possible implementation, determining the cross-entropy loss and distillation loss corresponding to the initial binary graph few-sample class-increasing model based on the class-increasing task set across multiple rounds during the class-increasing phase includes:

[0027] During the category augmentation phase, a preset number of categories are randomly selected from the training set corresponding to the category augmentation task set, and a preset number of samples are randomly selected from each category to form a support set and a query set.

[0028] The initial binary image few-sample class-incrementing model is used as the teacher controller, and the initial binary image few-sample class-incrementing model is copied as the current student controller;

[0029] The similarity between the samples in the support set and the query set is determined in the external memory unit by using the binary feature vectors obtained by the teacher controller, and the first prediction result of the query set is obtained.

[0030] The student controller determines the second prediction result for the query set by using samples from the support set and the query set.

[0031] Based on the first prediction result, the second prediction result, and the true category, the cross-entropy loss and distillation loss are determined.

[0032] In one possible implementation, the step of determining the predicted category of the test sample based on the test set using the binary graph few-sample class augmentation model during the inference phase includes:

[0033] During the inference phase, a preset number of categories are randomly selected from the test set, and a preset number of samples are randomly selected from each category to form a support set and a query set;

[0034] The similarity between the binary feature vector of the query set and the binary feature vector corresponding to the support set stored in the external memory unit is calculated by simulating a dot product, thereby determining the predicted category of the query set sample.

[0035] In one possible implementation, the binary graph few-sample class augmentation model employs a binary graph convolutional network. After binarizing the network weights and input nodes during the feature extraction stage, the graphs corresponding to the graph dataset are aggregated.

[0036] In one possible implementation, after storing the first binary feature vector and the tag corresponding to the external memory unit in the external memory unit, the following steps are included:

[0037] During retrieval, the dot product is used to calculate the similarity between the current binary feature vector and the corresponding first binary feature vectors in the external memory unit.

[0038] The label of the first binary feature vector with the highest similarity among multiple similarities is determined as the label corresponding to the current binary feature vector.

[0039] Secondly, the present invention also provides an in-memory computation implementation apparatus for graph few-sample class augmentation learning, the apparatus comprising:

[0040] The first partitioning module is used to divide the graph dataset into multiple rounds of class-adding task sets according to categories, wherein each round adds a preset number of categories compared to the previous round;

[0041] The second partitioning module is used to divide each of the aforementioned class-addition task sets into a training set and a test set;

[0042] The first training module is used to train an initial binary graph small sample class-increasing model based on the training set corresponding to the class-increasing task set in the 0th round during the basic stage.

[0043] The second training module is used to train a binary graph small sample class-increasing model based on the class-increasing task set of multiple rounds combined with the initial binary graph small sample class-increasing model during the class-increasing phase.

[0044] The determination module is used to determine the predicted category of the test sample based on the test set using the binary graph few-sample class augmentation model during the inference phase.

[0045] In one possible implementation, the binary graph few-shot class augmentation model includes interconnected controllers and external memory units, and the first training module includes:

[0046] The first determining submodule is used, in the basic phase, to determine the error between the predicted class and the true class based on the training set corresponding to the class augmentation task set in round 0;

[0047] The second determining submodule is used to update the parameters in the controller based on the error and determine the initial binary image small sample class augmentation model that has been trained.

[0048] In one possible implementation, the second training module includes:

[0049] The third determination submodule is used to determine the cross-entropy loss and distillation loss corresponding to the initial binary graph small sample class-increasing model based on the class-increasing task set of multiple rounds during the class-increasing phase.

[0050] The fourth determination submodule is used to update the control parameters of the binary small sample class-increasing model obtained by copying the initial binary small sample class-increasing model based on the cross-entropy loss and the distillation loss, and to determine the trained binary small sample class-increasing model.

[0051] In one possible implementation, the first determining submodule includes:

[0052] The first selection unit is used to randomly select a preset number of categories from the initial training set corresponding to the class-addition task set corresponding to the 0th round in the basic stage, and randomly select a preset number of samples for each category to form a support set and a query set.

[0053] An acquisition unit is used to acquire a first binary feature vector from the samples in the support set through the controller;

[0054] A storage unit is used to store the first binary feature vector and the tag corresponding to the external memory unit in the external memory unit;

[0055] The first determining unit is used to determine the similarity between the second binary feature vector obtained by the controller for the samples of the query set and the first binary feature vector;

[0056] The second determining unit is used to determine the predicted category of the query set based on the similarity.

[0057] The third determining unit is used to determine the error between the predicted category and the true category.

[0058] In one possible implementation, the third determining submodule includes:

[0059] The second selection unit is used to randomly select a preset number of categories from the training set corresponding to the category addition task set during the category addition stage, and randomly select a preset number of samples for each category to form a support set and a query set.

[0060] The copying unit is used to use the initial binary image few-sample class-incrementing model as the teacher controller and to copy the initial binary image few-sample class-incrementing model as the current student controller.

[0061] The fourth determining unit is used to determine the similarity between the samples in the support set and the query set through the binary feature vector obtained by the teacher controller in the external memory unit, and obtain the first prediction result of the query set.

[0062] The fifth determining unit is used to determine the second prediction result of the query set by using the samples in the support set and the query set through the student controller;

[0063] The sixth determining unit is used to determine the cross-entropy loss and distillation loss based on the first prediction result, the second prediction result, and the true category.

[0064] In one possible implementation, the determining module includes:

[0065] The selection submodule is used to randomly select a preset number of categories from the test set during the inference phase, and to randomly select a preset number of samples from each category to form a support set and a query set.

[0066] The fifth determination submodule is used to calculate the similarity between the binary feature vector of the query set and the binary feature vector corresponding to the support set stored in the external memory unit in a simulated dot product manner, and to determine the predicted category of the query set sample.

[0067] In one possible implementation, the binary graph few-sample class augmentation model employs a binary graph convolutional network. After binarizing the network weights and input nodes during the feature extraction stage, the graphs corresponding to the graph dataset are aggregated.

[0068] In one possible implementation, the device further includes:

[0069] The query unit is used to calculate multiple similarities between the current binary feature vector and the corresponding multiple first binary feature vectors in the external memory unit when performing a retrieval.

[0070] The seventh determining unit is used to determine the label of the first binary feature vector with the highest similarity among multiple similarities as the label corresponding to the current binary feature vector.

[0071] The beneficial effects of the in-memory computation implementation device for graph few-sample class incremental learning provided in the second aspect are the same as those of the in-memory computation implementation method for graph few-sample class incremental learning described in the first aspect or any possible implementation of the first aspect, and will not be repeated here.

[0072] Thirdly, the present invention also provides an electronic device, comprising: one or more processors; and one or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, cause to perform an in-memory computation implementation method for graph few-sample class augmentation learning described in any possible implementation of the first aspect.

[0073] The beneficial effects of the electronic device provided in the third aspect are the same as those of the in-memory computation implementation device for graph small sample class augmentation described in the second aspect or any possible implementation of the second aspect, and will not be elaborated here. Attached Figure Description

[0074] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0075] Figure 1 The diagram illustrates a flowchart of an in-memory computation implementation method for graph small-sample class augmentation learning provided in an embodiment of this application.

[0076] Figure 2 This paper illustrates a flowchart of another in-memory computation implementation method for graph small-sample class augmentation learning provided in an embodiment of this application.

[0077] Figure 3 This illustration shows a structural schematic diagram of a binary graph few-sample class incremental model (Bi-FSCIL) provided in an embodiment of this application;

[0078] Figure 4 This illustration shows a schematic diagram of an in-memory computing hardware implementation structure for graph small-sample class augmentation learning provided in an embodiment of this application;

[0079] Figure 5 This illustration shows a schematic diagram of a prediction accuracy test provided in an embodiment of this application;

[0080] Figure 6 This paper shows a structural flowchart of an in-memory computing implementation device for small-sample class augmentation learning provided in an embodiment of this application;

[0081] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention;

[0082] Figure 8 This is a schematic diagram of the chip structure provided in an embodiment of the present invention. Detailed Implementation

[0083] To facilitate a clear description of the technical solutions in the embodiments of the present invention, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. For example, the first threshold and the second threshold are merely used to distinguish different thresholds and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" are not necessarily different.

[0084] It should be noted that in this invention, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0085] In this invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a combination of a and b, a combination of a and c, a combination of b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0086] Figure 1 This illustration shows a flowchart of an in-memory computation implementation method for graph few-sample class augmentation learning provided in an embodiment of this application. Figure 1 As shown, the in-memory computation implementation method for small sample class augmentation learning includes:

[0087] Step 101: Divide the graph dataset into multiple rounds of class-addition task sets according to categories, where each round adds a preset number of categories compared to the previous round.

[0088] In this application, the graph dataset can be divided into multiple rounds of class-based incremental task sets (D0, D1, ..., D...) according to the categories. T In this embodiment, N new categories are added in each round compared to the previous round. The specific number of new categories added in each round is not limited in this embodiment, and can be set according to the actual application scenario.

[0089] Step 102: Divide each of the above-mentioned incremental task sets into a training set and a test set.

[0090] In this application, each class-addition task set can be divided into a training set and a test set, and the controller update adopts an optimization-based meta-learning method, including a basic phase and a class-addition phase.

[0091] Step 103: In the basic stage, an initial binary graph few-sample class-increment model is trained based on the training set corresponding to the class-increment task set described in round 0.

[0092] In this application, a meta-training method is used in the basic stage, that is, the model is pre-trained using multiple few-shot learning tasks and the weights are initialized using prior knowledge.

[0093] Step 104: In the class augmentation stage, the binary graph few-shot class augmentation model is trained based on the class augmentation task set from multiple rounds and the initial binary graph few-shot class augmentation model.

[0094] In this application, a meta-fine-tuning method with knowledge distillation is used in the class addition stage. The role of knowledge distillation is to transfer the feature extraction capabilities of the previous teacher controller to the new student controller, so as to avoid forgetting the previously learned classes when new classes appear.

[0095] Step 105: In the inference phase, the predicted category of the test sample is determined based on the test set using the binary graph few-sample class augmentation model.

[0096] During the inference phase, a preset number of categories are randomly selected from the test set, and a preset number of samples are randomly selected from each category to form a support set and a query set. The similarity between the binary feature vector of the query set and the corresponding binary feature vector of the support set stored in the external memory unit is calculated using a simulated dot product to determine the predicted category of the query set samples.

[0097] The in-memory computation implementation method for graph few-shot class augmentation learning provided in this invention divides the graph dataset into multiple rounds of class augmentation task sets according to categories, wherein each round adds a preset number of categories compared to the previous round; each class augmentation task set is divided into a training set and a test set; in the basic stage, an initial binary graph few-shot class augmentation model is trained based on the training set corresponding to the class augmentation task set in round 0; in the class augmentation stage, a binary graph few-shot class augmentation model is trained based on the class augmentation task sets in multiple rounds combined with the initial binary graph few-shot class augmentation model; in the inference stage, the predicted category of the test sample is determined based on the test set using the binary graph few-shot class augmentation model. This method can achieve high end-to-end node classification accuracy and can effectively accelerate the convergence of the training process, reducing the number of programming iterations.

[0098] Optional, Figure 2 This illustration shows a flowchart of another in-memory computation implementation method for graph few-sample class augmentation learning provided in an embodiment of this application. See also... Figure 2 The in-memory computation implementation methods for small sample class augmentation learning in this graph include:

[0099] Step 201: Divide the graph dataset into multiple rounds of class-adding task sets according to categories, where each round adds a preset number of categories compared to the previous round.

[0100] In this application, the in-memory computation method for graph few-shot incremental learning adopts a binary graph few-shot incremental model. Figure 3 This illustration shows a structural schematic diagram of a binary graph few-sample class incremental model (Bi-FSCIL) provided in an embodiment of this application, as shown below. Figure 3 As shown, the model includes an interconnected controller 01 and an external memory unit 02. The binary graph few-sample class augmentation model employs a binary graph convolutional network. After binarizing the network weights and input nodes during the feature extraction stage, it performs graph aggregation processing on the graph dataset.

[0101] The feature extraction process is shown in equation (1) below, which describes the binary node features of the (l+1)th layer. Represented as:

[0102]

[0103] in, β represents the binary node features of the l-th layer. (l) W represents the average scaling factor of the binary node feature. B (l) Let α represent the binary weight matrix of the first layer. (l) The average scaling factor of the weight matrix, and the adjacency matrix of the graph. Multiplying by the feature extraction result Z yields a layer 1 node feature that aggregates neighborhood information. After passing through the nonlinear activation function σ, a layer 1+1 binary node feature is obtained.

[0104] The binarization of node features and weights uses a sign function. Taking node features as an example, the binarization process is shown in equation (2):

[0105]

[0106] Among them, F i,: Represent the real-valued feature of node i. and β i Let β represent the binary node features and average scaling factor of node i, respectively. i The 1-norm ||F| of the real-valued features of node i i,: ||1 divided by the number of nodes N, F is obtained. i,: After passing through the sign function, it becomes a binary feature containing only {+1,-1}.

[0107] In this application, the graph dataset can be divided into multiple rounds of class-based incremental task sets (D0, D1, ..., D...) according to the categories. T In this embodiment, N new categories are added in each round compared to the previous round. The specific number of new categories added in each round is not limited in this embodiment, and can be set according to the actual application scenario.

[0108] Step 202: Divide each of the above-mentioned incremental task sets into a training set and a test set.

[0109] In this application, each class-addition task set can be divided into a training set and a test set, and the controller update adopts an optimization-based meta-learning method, including a basic phase and a class-addition phase.

[0110] Step 203: In the basic phase, determine the error between the predicted class and the true class based on the training set corresponding to the class augmentation task set described in round 0.

[0111] In this application, a meta-training method is used in the basic stage, that is, the model is pre-trained using multiple few-shot learning tasks and the weights are initialized using prior knowledge. The specific implementation process of this stage may include the following sub-steps:

[0112] Sub-step A1: In the basic stage, a preset number of categories are randomly selected from the initial training set corresponding to the class-addition task set corresponding to round 0, and a preset number of samples are randomly selected from each category to form a support set and a query set.

[0113] Among them, m categories are randomly selected from the training set corresponding to D0, and n samples are randomly selected from each category to form a support set and a query set.

[0114] Sub-step A2: Obtain the first binary feature vector from the samples in the support set through the controller.

[0115] The feature extraction process is shown in equation (1) below, which describes the binary node features of the (l+1)th layer. Represented as:

[0116]

[0117] in, β represents the binary node features of the l-th layer. (l) W represents the average scaling factor of the binary node feature. B (l) Let α represent the binary weight matrix of the first layer. (l) The average scaling factor of the weight matrix, and the adjacency matrix of the graph. The extracted feature result Z yielded a layer 1 node feature that aggregated neighborhood information. After passing through the nonlinear activation function σ, a layer 1+1 binary node feature was obtained.

[0118] The binarization of node features and weights uses a sign function. Taking node features as an example, the binarization process is shown in equation (2):

[0119]

[0120] Among them, F i,: Represent the real-valued feature of node i. and β i Let β represent the binary node features and average scaling factor of node i, respectively. i The 1-norm ||F| of the real-valued features of node i i,: ||1 divided by the number of nodes N, F is obtained. i,: After passing through the sign function, it becomes a binary feature containing only {+1,-1}.

[0121] Sub-step A3: Store the first binary feature vector and the tag corresponding to the external memory unit in the external memory unit.

[0122] Sub-step A4: Determine the similarity between the second binary feature vector obtained by the controller for the samples of the query set and the first binary feature vector.

[0123] Sub-step A5: Determine the predicted category p of the query set based on the similarity. θThat is, the probability value that the query set sample belongs to different categories.

[0124] Sub-step A6: Determine the error between the predicted category and the true category y.

[0125] Optionally, the error is calculated using the cross-entropy method shown in equation (3):

[0126] ι cross-entropy =Σ-ylogp θ Equation (3).

[0127] Following sub-step A3, the following is also included:

[0128] Sub-step A7: During the retrieval, the dot product is used to calculate the similarity between the current binary feature vector and the corresponding first binary feature vectors in the external memory unit;

[0129] Sub-step A8: Determine the label of the first binary feature vector with the highest similarity among multiple similarities as the label corresponding to the current binary feature vector.

[0130] Step 204: Update the parameters in the controller based on the error to determine the initial binary image small sample class augmentation model that has been trained.

[0131] Step 205: In the class augmentation phase, determine the cross-entropy loss and distillation loss corresponding to the initial binary graph small sample class augmentation model based on the class augmentation task set of multiple rounds.

[0132] In this application, a meta-fine-tuning method with knowledge distillation is used in the class addition stage. The role of knowledge distillation is to transfer the feature extraction capabilities of the previous teacher controller to the new student controller, so as to avoid forgetting the previously learned classes when new classes appear.

[0133] The specific implementation of step 205 above may include the following sub-steps:

[0134] Sub-step B1: In the class addition phase, a preset number of classes are randomly selected from the training set corresponding to the class addition task set, and a preset number of samples are randomly selected from each class to form a support set and a query set.

[0135] Sub-step B2: Use the initial binary graph few-sample class-incrementing model as the teacher controller, and copy the initial binary graph few-sample class-incrementing model as the current student controller.

[0136] Sub-step B3: Using the binary feature vectors obtained by the teacher controller, the samples in the support set and the query set are used to determine the similarity in the external memory unit, thus obtaining the first prediction result p of the query set. θ-1 .

[0137] Sub-step B4: Using the student controller, determine the second prediction result p of the query set by combining the samples from the support set and the query set. θ .

[0138] Sub-step B5: Determine the cross-entropy loss and distillation loss based on the first prediction result, the second prediction result, and the true category.

[0139] Specifically, using equation (4), the cross-entropy loss and distillation loss can be calculated based on the first prediction result, the second prediction result, and the true label y.

[0140] l knowledge-distillation =∑[KL(p θ-1 ||p θ Equation (4).

[0141] Step 206: Update the control parameters of the binary graph small sample class augmentation model obtained by copying the initial binary graph small sample class augmentation model based on the cross-entropy loss and the distillation loss, and determine the trained binary graph small sample class augmentation model.

[0142] The control parameters are the weight parameters of the binary graph convolutional neural network, which can be updated using backpropagation.

[0143] Step 207: In the inference phase, a preset number of categories are randomly selected from the test set, and a preset number of samples are randomly selected from each category to form a support set and a query set.

[0144] Step 208: Calculate the similarity between the binary feature vector of the query set and the binary feature vector corresponding to the support set stored in the external memory unit using a simulated dot product method to determine the predicted category of the query set sample.

[0145] Figure 4 This illustration shows a schematic diagram of an in-memory computing hardware implementation structure for graph small-sample class augmentation learning provided in an embodiment of this application, as shown below. Figure 4As shown, the controller weights and graph node features can first be binarized. Then, by setting or resetting the resistive random access memory (RRAM), the controller's binary weights are physically mapped onto the in-memory computing (IMC) chip array, and the binary node features are mapped to voltage inputs. The feature extraction process in the controller is achieved through analog multiplication and accumulation operations on the RRAM array on the IMC chip. Similarly, the binary states of the RRAM device can be used to store feature vectors in external memory, and the dot product distance between the query vector and the stored feature vector can be calculated on the RRAM array. In each new round, a new class vector is stored in the new space of the external memory. The node classification accuracy of the RRAM-based binary graph few-shot class augmentation model (Bi-FSCIL) is comparable to that of the software-based node classification model. At the same time, the meta-fine-tuning method with distillation effectively accelerates the convergence of the training process, thereby reducing the number of RRAM programming operations.

[0146] In this application, the graph dataset is the CORA graph dataset, in which each sample point represents a scientific paper and belongs to seven categories: genetic algorithm, neural network, probabilistic method, reinforcement learning, etc. Figure 5 This illustration shows a prediction accuracy test diagram provided by an embodiment of this application, such as... Figure 5 As shown, the horizontal axis represents the streaming session, including 0 (Base stage), 1, 2, 3, 4 and 5, and the vertical axis represents the accuracy. It can be determined from the figure that the end-to-end node classification accuracy of Bi-GFSCIL in RRAM is comparable to that of node classification accuracy using software (GPU).

[0147] The in-memory computation implementation method for graph few-shot class augmentation learning provided in this invention divides the graph dataset into multiple rounds of class augmentation task sets according to categories, wherein each round adds a preset number of categories compared to the previous round; each class augmentation task set is divided into a training set and a test set; in the basic stage, an initial binary graph few-shot class augmentation model is trained based on the training set corresponding to the class augmentation task set in round 0; in the class augmentation stage, a binary graph few-shot class augmentation model is trained based on the class augmentation task sets in multiple rounds combined with the initial binary graph few-shot class augmentation model; in the inference stage, the predicted category of the test sample is determined based on the test set using the binary graph few-shot class augmentation model. This method can achieve high end-to-end node classification accuracy and can effectively accelerate the convergence of the training process, reducing the number of programming iterations.

[0148] Figure 6 This illustration shows a schematic diagram of an in-memory computing implementation device for small-sample class augmentation learning provided in an embodiment of this application. Figure 6As shown, the in-memory computing implementation device 300 for small sample class augmentation learning includes:

[0149] The first partitioning module 301 is used to partition the graph dataset into a multi-round class-addition task set according to the categories, wherein each round adds a preset number of categories compared to the previous round;

[0150] The second partitioning module 302 is used to partition each of the class-addition task sets into a training set and a test set;

[0151] The first training module 303 is used to train an initial binary graph small sample class-increasing model based on the training set corresponding to the class-increasing task set in the 0th round during the basic stage.

[0152] The second training module 304 is used to train a binary graph small sample class-increasing model based on the class-increasing task set of multiple rounds and the initial binary graph small sample class-increasing model during the class-increasing phase.

[0153] The determination module 305 is used to determine the predicted category of the test sample based on the test set using the binary graph few-sample class augmentation model during the inference phase.

[0154] Optionally, the binary graph few-shot class augmentation model includes interconnected controllers and external memory units, and the first training module includes:

[0155] The first determining submodule is used, in the basic phase, to determine the error between the predicted class and the true class based on the training set corresponding to the class augmentation task set in round 0;

[0156] The second determining submodule is used to update the parameters in the controller based on the error and determine the initial binary image small sample class augmentation model that has been trained.

[0157] Optionally, the second training module includes:

[0158] The third determination submodule is used to determine the cross-entropy loss and distillation loss corresponding to the initial binary graph small sample class-increasing model based on the class-increasing task set of multiple rounds during the class-increasing phase.

[0159] The fourth determination submodule is used to update the control parameters of the binary small sample class-increasing model obtained by copying the initial binary small sample class-increasing model based on the cross-entropy loss and the distillation loss, and to determine the trained binary small sample class-increasing model.

[0160] Optionally, the first determining submodule includes:

[0161] The first selection unit is used to randomly select a preset number of categories from the initial training set corresponding to the class-addition task set corresponding to the 0th round in the basic stage, and randomly select a preset number of samples for each category to form a support set and a query set.

[0162] An acquisition unit is used to acquire a first binary feature vector from the samples in the support set through the controller;

[0163] A storage unit is used to store the first binary feature vector in the external memory unit;

[0164] The first determining unit is used to determine the similarity between the second binary feature vector obtained by the controller for the samples of the query set and the first binary feature vector;

[0165] The second determining unit is used to determine the predicted category of the query set based on the similarity.

[0166] The third determining unit is used to determine the error between the predicted category and the true category.

[0167] Optionally, the third determining submodule includes:

[0168] The second selection unit is used to randomly select a preset number of categories from the training set corresponding to the category addition task set during the category addition stage, and randomly select a preset number of samples for each category to form a support set and a query set.

[0169] The copying unit is used to use the initial binary image few-sample class-incrementing model as the teacher controller and to copy the initial binary image few-sample class-incrementing model as the current student controller.

[0170] The fourth determining unit is used to determine the similarity between the samples in the support set and the query set through the binary feature vector obtained by the teacher controller in the external memory unit, and obtain the first prediction result of the query set.

[0171] The fifth determining unit is used to determine the second prediction result of the query set by using the samples in the support set and the query set through the student controller;

[0172] The sixth determining unit is used to determine the cross-entropy loss and distillation loss based on the first prediction result, the second prediction result, and the true category.

[0173] Optionally, the determining module includes:

[0174] The selection submodule is used to randomly select a preset number of categories from the test set during the inference phase, and to randomly select a preset number of samples from each category to form a support set and a query set.

[0175] The fifth determination submodule is used to calculate the similarity between the binary feature vector of the query set and the binary feature vector corresponding to the support set stored in the external memory unit in a simulated dot product manner, and to determine the predicted category of the query set sample.

[0176] Optionally, the binary graph few-sample class augmentation model employs a binary graph convolutional network. After binarizing the network weights and input nodes during the feature extraction stage, the graphs corresponding to the graph dataset are aggregated.

[0177] Optionally, the device further includes:

[0178] The query unit is used to calculate multiple similarities between the current binary feature vector and the corresponding multiple first binary feature vectors in the external memory unit when performing a retrieval.

[0179] The seventh determining unit is used to determine the label of the first binary feature vector with the highest similarity among multiple similarities as the label corresponding to the current binary feature vector.

[0180] The in-memory computation implementation device for graph few-shot class augmentation learning provided in this embodiment of the invention can, through a first partitioning module, divide the graph dataset into multiple rounds of class augmentation task sets according to categories, wherein each round adds a preset number of categories compared to the previous round; through a second partitioning module, each class augmentation task set is divided into a training set and a test set; through a first training module, in the basic stage, an initial binary graph few-shot class augmentation model is trained based on the training set corresponding to the class augmentation task set in round 0; through a second training module, in the class augmentation stage, a binary graph few-shot class augmentation model is trained based on the class augmentation task sets in multiple rounds combined with the initial binary graph few-shot class augmentation model; through a determination module, in the inference stage, the predicted category of the test sample is determined based on the test set using the binary graph few-shot class augmentation model. This can achieve high end-to-end node classification accuracy and effectively accelerate the convergence of the training process, reducing the number of programming iterations.

[0181] This invention provides an in-memory computation implementation device for graph few-sample class incremental learning, applied to a system including a controller and at least one detection circuit electrically connected to the controller, such as... Figures 1 to 5 The in-memory computation implementation method for incremental learning of any of the graphs shown is not described in detail here to avoid repetition.

[0182] The electronic device in this embodiment of the invention can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, a mobile electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc., while a non-mobile electronic device can be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This embodiment of the invention does not impose specific limitations.

[0183] The electronic device in this embodiment of the invention can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment of the invention does not impose specific limitations.

[0184] Figure 7 A schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention is shown. Figure 7 As shown, the electronic device 400 includes a processor 410.

[0185] like Figure 7 As shown, the processor 410 described above can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention.

[0186] like Figure 7 As shown, the electronic device 400 may further include a communication line 440. The communication line 440 may include a path for transmitting information between the components.

[0187] Optional, such as Figure 7 As shown, the above-described electronic device may further include a communication interface 420. There may be one or more communication interfaces 420. The communication interface 420 may use any transceiver-like device for communicating with other devices or communication networks.

[0188] Optional, such as Figure 7As shown, the electronic device may further include a memory 430. The memory 430 stores computer execution instructions for implementing the present invention, and its execution is controlled by a processor. The processor executes the computer execution instructions stored in the memory to implement the method provided in the embodiments of the present invention.

[0189] like Figure 7 As shown, memory 430 can be read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 430 can exist independently and be connected to processor 410 via communication line 440. Memory 430 can also be integrated with processor 410.

[0190] Optionally, the computer execution instructions in the embodiments of the present invention may also be referred to as application code, and the embodiments of the present invention do not specifically limit this.

[0191] In a specific implementation, as one example, such as Figure 7 As shown, processor 410 may include one or more CPUs, such as Figure 7 CPU0 and CPU1 in the CPU.

[0192] In a specific implementation, as one example, such as Figure 7 As shown, the terminal device may include multiple processors, such as Figure 7 The first processor 4101 and the second processor 4102 are included. Each of these processors can be a single-core processor or a multi-core processor.

[0193] Figure 8 This is a schematic diagram of the chip structure provided in an embodiment of the present invention. Figure 8 As shown, the chip 500 includes one or more processors 410.

[0194] Optional, such as Figure 8As shown, the chip also includes a communication interface 420 and a memory 430. The memory 430 may include read-only memory and random access memory, and provides operation instructions and data to the processor. A portion of the memory may also include non-volatile random access memory (NVRAM).

[0195] In some implementations, such as Figure 8 As shown, memory 430 stores the following elements: execution modules or data structures, or subsets thereof, or extended sets thereof.

[0196] In embodiments of the present invention, such as Figure 8 As shown, the corresponding operation is executed by calling the operation instructions stored in the memory (which can be stored in the operating system).

[0197] like Figure 8 As shown, the processor 410 controls the processing operations of any one of the terminal devices. The processor 410 can also be called a central processing unit (CPU).

[0198] like Figure 8 As shown, memory 430 may include read-only memory and random access memory, providing instructions and data to the processor. A portion of memory 430 may also include NVRAM. For example, in an application, memory, communication interfaces, and memory are coupled together via a bus system, which may include, in addition to a data bus, a power bus, a control bus, and a status signal bus, etc. However, for clarity, in... Figure 8 The general labeled all buses as Bus System 540.

[0199] like Figure 8As shown, the methods disclosed in the above embodiments of the present invention can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0200] On the one hand, a computer-readable storage medium is provided, which stores instructions that, when executed, implement the functions performed by the terminal device in the above embodiments.

[0201] On the one hand, a chip is provided that is used in a terminal device. The chip includes at least one processor and a communication interface. The communication interface and at least one processor are coupled together. The processor is used to run instructions to implement the functions executed by the in-memory computation implementation method of graph small sample class incremental learning in the above embodiments.

[0202] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are performed entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a terminal, a user equipment, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD).

[0203] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, the disclosure, and the appended claims in carrying out the claimed invention. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple components. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results. Although the invention has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made without departing from the spirit and scope of the invention. Accordingly, this specification and drawings are merely exemplary descriptions of the invention as defined by the appended claims and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the invention. Obviously, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if such modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include such modifications and modifications.

Claims

1. A method for in-memory computation of graph few-sample class augmentation learning, characterized in that, The method includes: The graph dataset is divided into multiple rounds of class-adding task sets according to categories, where each round adds a preset number of categories compared to the previous round; Each of the aforementioned class-specific task sets is divided into a training set and a test set; In the basic stage, an initial binary graph small sample class-increasing model is trained based on the training set corresponding to the class-increasing task set described in round 0. In the class augmentation phase, a binary graph few-shot class augmentation model is trained based on the class augmentation task set from multiple rounds and the initial binary graph few-shot class augmentation model; the binary graph few-shot class augmentation model includes interconnected controllers and external memory units; During the inference phase, the predicted category of the test sample is determined based on the test set using the binary graph few-sample class augmentation model; In the class augmentation phase, a binary graph few-shot class augmentation model is trained based on the class augmentation task set from multiple rounds and the initial binary graph few-shot class augmentation model, including: During the class augmentation phase, the cross-entropy loss and distillation loss corresponding to the initial binary graph small sample class augmentation model are determined based on the class augmentation task set of multiple rounds. Based on the cross-entropy loss and the distillation loss, update the control parameters of the binary small sample class-increasing model obtained by copying the initial binary small sample class-increasing model, and determine the trained binary small sample class-increasing model; In the class-incrementing phase, the cross-entropy loss and distillation loss corresponding to the initial binary graph few-sample class-incrementing model are determined based on the class-incrementing task set over multiple rounds, including: During the category augmentation phase, a preset number of categories are randomly selected from the training set corresponding to the category augmentation task set, and a preset number of samples are randomly selected from each category to form a support set and a query set. The initial binary image few-sample class-incrementing model is used as the teacher controller, and the initial binary image few-sample class-incrementing model is copied as the current student controller; The similarity between the samples in the support set and the query set is determined in the external memory unit by using the binary feature vectors obtained by the teacher controller, and the first prediction result of the query set is obtained. The student controller uses samples from the support set and the query set to determine the second prediction result for the query set. Based on the first prediction result, the second prediction result, and the true category, the cross-entropy loss and distillation loss are determined.

2. The method according to claim 1, characterized in that, In the basic stage, an initial binary graph few-sample class-increasing model is trained based on the training set corresponding to the class-increasing task set in round 0, including: In the basic phase, the error between the predicted class and the true class is determined based on the training set corresponding to the class augmentation task set described in round 0; Based on the error, the parameters in the controller are updated to determine the initial binary image small sample class augmentation model that has been trained.

3. The method according to claim 2, characterized in that, In the basic phase, determining the error between the predicted class and the true class based on the training set corresponding to the class-addition task set in round 0 includes: In the basic stage, a preset number of categories are randomly selected from the initial training set corresponding to the class-addition task set corresponding to the 0th round, and a preset number of samples are randomly selected from each category to form a support set and a query set; The first binary feature vector is obtained from the samples in the support set through the controller; The first binary feature vector and the tag corresponding to the external memory unit are stored in the external memory unit; Determine the similarity between the second binary feature vector obtained by the controller and the first binary feature vector of the samples in the query set; The predicted category of the query set is determined based on the similarity. Determine the error between the predicted category and the true category.

4. The method according to claim 2, characterized in that, In the inference phase, the predicted category of the test sample is determined based on the test set using the binary graph few-sample class augmentation model, including: During the inference phase, a preset number of categories are randomly selected from the test set, and a preset number of samples are randomly selected from each category to form a support set and a query set; The similarity between the binary feature vector of the query set and the binary feature vector corresponding to the support set stored in the external memory unit is calculated by simulating a dot product, thereby determining the predicted category of the query set sample.

5. The method according to claim 3, characterized in that, The binary graph few-sample class augmentation model employs a binary graph convolutional network. After binarizing the network weights and input nodes during the feature extraction stage, it performs graph aggregation processing on the graph dataset.

6. The method according to claim 5, characterized in that, After storing the first binary feature vector and the tag corresponding to the external memory unit in the external memory unit, the process includes: During retrieval, the dot product is used to calculate the similarity between the current binary feature vector and the corresponding first binary feature vectors in the external memory unit. The label of the first binary feature vector with the highest similarity among multiple similarities is determined as the label corresponding to the current binary feature vector.

7. An in-memory computational implementation device for graph small-sample class augmentation learning, characterized in that, The device includes: The first partitioning module is used to divide the graph dataset into multiple rounds of class-adding task sets according to categories, wherein each round adds a preset number of categories compared to the previous round; The second partitioning module is used to divide each of the aforementioned class-addition task sets into a training set and a test set; The first training module is used to train an initial binary graph small sample class-increasing model based on the training set corresponding to the class-increasing task set in the 0th round during the basic stage. The second training module is used to train a binary graph few-shot class-increasing model based on the class-increasing task set from multiple rounds and the initial binary graph few-shot class-increasing model during the class-increasing phase; the binary graph few-shot class-increasing model includes interconnected controllers and external memory units. The determination module is used to determine the predicted category of the test sample based on the test set using the binary graph few-sample class augmentation model during the inference phase. The second training module includes: The third determination submodule is used to determine the cross-entropy loss and distillation loss corresponding to the initial binary graph small sample class-increasing model based on the class-increasing task set of multiple rounds during the class-increasing phase. The fourth determination submodule is used to update the control parameters of the binary graph small sample class-increasing model obtained by copying the initial binary graph small sample class-increasing model based on the cross-entropy loss and the distillation loss, and to determine the trained binary graph small sample class-increasing model. The third determining submodule includes: The second selection unit is used to randomly select a preset number of categories from the training set corresponding to the category addition task set during the category addition stage, and randomly select a preset number of samples for each category to form a support set and a query set. The copying unit is used to use the initial binary image few-sample class-incrementing model as the teacher controller and to copy the initial binary image few-sample class-incrementing model as the current student controller. The fourth determining unit is used to determine the similarity between the samples in the support set and the query set through the binary feature vector obtained by the teacher controller in the external memory unit, and obtain the first prediction result of the query set. The fifth determining unit is used to determine the second prediction result of the query set by using the student controller to combine the samples in the support set and the query set. The sixth determining unit is used to determine the cross-entropy loss and distillation loss based on the first prediction result, the second prediction result, and the true category.

8. An electronic device, characterized in that, include: One or more processors; And one or more machine-readable media thereon storing instructions, which, when executed by the one or more processors, cause the in-memory computation implementation method of graph few-sample class augmentation learning as described in any one of claims 1-6 to be performed.

Images