Acceleration method and device of full binary graph convolutional network based on RRAM and electronic equipment
Patent Information
- Application Number
- CN202311355365.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-18
- Publication Date
- 2026-11-03
- Estimated Expiration
- 2043-10-18
AI Technical Summary
The large number of model parameters in graph convolutional networks, coupled with the non-ideal nature of RRAM devices, means that the hardware implementation of graph convolutional networks still faces challenges in terms of the scale and accuracy of RRAM arrays.
In the combination and aggregation stage of the binary graph convolutional network, a binarization method is adopted. Features are extracted through fully connected layers with binary weights and binarized using the Sign function. Matrix-vector multiplication and content-addressable memory operations are performed using an RRAM array to optimize node features.
While maintaining the recognition rate, the model parameters and graph data memory are greatly reduced, computational power consumption is lowered, and hardware implementation efficiency is improved.
Smart Images
Figure CN117634565B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of machine learning and artificial intelligence, and in particular to an acceleration method, apparatus and electronic device for RRAM-based fully binary graph convolutional networks. Background Technology
[0002] Graph data is ubiquitous in the real world and has a wide range of applications. With the rapid development of the Internet of Things and cloud computing, the scale and complexity of graph data are increasing rapidly, posing a significant challenge to current graph acceleration hardware systems. Graph Convolutional Networks (GCNs) have been widely used to process, analyze, and learn efficient representations of graph data. Each layer of a GCN mainly consists of two phases: an aggregation phase and a combination phase.
[0003] During the aggregation phase, neighbor node features propagate along the edges and are aggregated at each node. Because graph data exhibits a power-law distribution, the number and location of neighbors vary greatly between nodes. Therefore, the memory access pattern during the aggregation phase is dynamic and irregular. In the combination phase, optimized fully connected layers are used to project the aggregated node features onto a new subspace. This operation is typically represented by matrix-vector multiplication and exhibits both static and regular computation and memory access patterns. This mixed regular and irregular computation and memory access pattern in GCN leads to problems such as load imbalance, excessive memory access, and data processing complexity. General-purpose processors, such as CPUs and GPUs, cannot effectively fuse the execution of these two phases.
[0004] In-memory computing architectures based on emerging non-volatile memory devices, such as resistive random access memory (RRAM), have been proposed to accelerate deep neural networks. These accelerators significantly improve the energy efficiency of the combinational stage by mapping matrix-vector multiplication operations to multiply-and-accumulate (MAC) operations on RRAM arrays, avoiding data transfer between the processor and memory, i.e., the von Neumann bottleneck.
[0005] However, these accelerators cannot address the large amount of dynamic and irregular data access in the aggregation stage. Recently proposed graph accelerators combine Content Addressable Memory (CAM) operations with RRAM array-based MAC operations to address the irregularities in the aggregation stage by directly computing on the sparse data representation.
[0006] However, GCN has a large number of model parameters and RRAM devices are non-ideal, which means that the hardware implementation of GCN still faces challenges in terms of the scale and accuracy of RRAM arrays. Therefore, an acceleration method is urgently needed to solve the problem of high power consumption in existing computing. Summary of the Invention
[0007] The purpose of this invention is to provide an acceleration method, apparatus, and electronic device for fully binary graph convolutional networks based on RRAM, which solves the problem that the large number of model parameters of graph convolutional networks and the non-ideal nature of RRAM devices still pose challenges to the hardware implementation of graph convolutional networks in terms of the scale and accuracy of RRAM arrays.
[0008] In a first aspect, the present invention provides an acceleration method for fully binary graph convolutional networks based on RRAM, comprising:
[0009] In the binary combination stage of the Lth layer in the binary graph convolutional network, the node features of the graph data are extracted through a fully connected layer with binary weights and binarized to obtain the first binarized node features.
[0010] In the binary aggregation stage of the Lth layer in the binary graph convolutional network, the first binarized node features are aggregated and binarized based on the neighboring nodes of each node to obtain the updated node features;
[0011] The binary combination stage and the binary aggregation stage are iteratively performed on all nodes in the node set to obtain the optimized node features;
[0012] The graph data or nodes are classified based on the node features.
[0013] When the above technical solution is applied to a binary graph convolutional network, the method includes: in the binary combination stage of the Lth layer of the binary graph convolutional network, extracting node features of the graph data through a fully connected layer with binary weights and performing binarization processing to obtain first binarized node features; in the binary aggregation stage of the Lth layer of the binary graph convolutional network, aggregating and binarizing the first binarized node features based on the neighboring nodes of each node to obtain updated node features; iteratively performing the binary combination stage and the binary aggregation stage on all nodes in the node set to obtain optimized node features; and classifying graph data or nodes based on the node features, which can greatly reduce model parameters and graph data memory while ensuring recognition rate and reducing computational power consumption.
[0014] In one possible implementation, during the binary combination stage of the Lth layer in the binary graph convolutional network, the node features of the graph data are extracted through a fully connected layer with binary weights and binarized to obtain the first binary node features, including:
[0015] In the binary combination stage of the Lth layer in the binary graph convolutional network, the node features of the graph data are extracted through a fully connected layer with binary weights by combining the Sign function, and then binarized to obtain the first binary node features.
[0016] In one possible implementation, during the binary combination stage of the Lth layer in the binary graph convolutional network, the node features of the graph data are extracted through a fully connected layer with binary weights using the Sign function, and then binarized to obtain the first binary node features, including:
[0017] In the binary combination stage of the Lth layer in the binary graph convolutional network, the full-precision weights are binarized by the Sign function to obtain the binarized weights.
[0018] For each node, the node features of the previous layer are extracted by matrix-vector multiplication using the binarized weights;
[0019] The node features are binarized to obtain the first binarized node features.
[0020] In one possible implementation, during the binary aggregation stage of the Lth layer in the binary graph convolutional network, the first binarized node features are aggregated and binarized based on the neighboring nodes of each node to obtain updated node features, including:
[0021] In the binary aggregation stage of the Lth binary graph convolutional layer, the neighbor node features of each node and the first binary node features corresponding to the node are added together, the average is taken, and then binarization is performed to obtain the updated node features.
[0022] In one possible implementation, after classifying the graph data or nodes based on the node features, the method further includes:
[0023] During the backpropagation phase, the binary graph convolutional network is corrected using the cross-entropy loss function.
[0024] Secondly, the present invention also provides an acceleration device for a fully binary graph convolutional network based on RRAM, the device comprising:
[0025] The first binarization processing module is used to extract the node features of the graph data through a fully connected layer with binary weights and perform binarization processing in the binary combination stage of the Lth layer in the binary graph convolutional network to obtain the first binarized node features.
[0026] The second binarization processing module is used to perform aggregation and binarization processing on the first binarized node features based on the neighbor nodes of each node in the binary aggregation stage of the Lth layer of the binary graph convolutional network to obtain the updated node features.
[0027] The iteration module is used to iteratively perform the binary combination stage and the binary aggregation stage on all the nodes in the node set to obtain the optimized node features;
[0028] The classification module is used to classify graph data or nodes based on the node features.
[0029] In one possible implementation, the first binarization processing module includes:
[0030] The first binarization processing submodule is used to extract the node features of the graph data through a fully connected layer with binary weights in the binary combination stage of the Lth layer in the binary graph convolutional network, and perform binarization processing to obtain the first binarized node features.
[0031] The first binarization processing submodule includes:
[0032] The first binarization unit is used to binarize the full-precision weights through the Sign function in the binary combination stage of the Lth layer in the binary graph convolutional network to obtain binarized weights.
[0033] The node feature extraction unit is used to extract node features for each node by using the binarized weights through matrix-vector multiplication operations on the node features of the previous layer.
[0034] The second binarization unit is used to binarize the node features to obtain the first binarized node features.
[0035] In one possible implementation, the second binarization processing module includes:
[0036] The second binarization processing submodule is used in the binary aggregation stage of the Lth layer binary graph convolutional layer to add the neighbor node features of each node and the first binarized node features corresponding to the node, take the average, and then perform binarization processing to obtain the updated node features.
[0037] In one possible implementation, the device further includes:
[0038] A correction module is used to correct the binary graph convolutional network using the cross-entropy loss function during the backpropagation phase.
[0039] The advantages of the acceleration device for RRAM-based fully binary graph convolutional networks provided in the second aspect are the same as those of the acceleration method for RRAM-based fully binary graph convolutional networks described in the first aspect or any possible implementation of the first aspect, and will not be repeated here.
[0040] Thirdly, the present invention also provides an electronic device comprising: one or more processors; and one or more machine-readable media having instructions stored thereon, which, when executed by said one or more processors, cause to perform an accelerated method for an RRAM-based fully binary graph convolutional network as described in any possible implementation of the first aspect.
[0041] The beneficial effects of the electronic device provided in the third aspect are the same as those of the acceleration method for RRAM-based fully binary graph convolutional networks described in the first aspect or any possible implementation of the first aspect, and will not be repeated here. Attached Figure Description
[0042] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:
[0043] Figure 1 The diagram shows a flowchart of an acceleration method for a fully binary graph convolutional network based on RRAM, according to an embodiment of this application.
[0044] Figure 2 This paper illustrates a flowchart of another acceleration method for a fully binary graph convolutional network based on RRAM, provided in an embodiment of this application.
[0045] Figure 3 This illustration shows a schematic diagram of a hardware architecture for accelerating a fully binary graph convolutional network based on RRAM, according to an embodiment of this application.
[0046] Figure 4 A flowchart illustrating the structure of an acceleration device for a fully binary graph convolutional network based on RRAM, according to an embodiment of this application, is shown.
[0047] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention;
[0048] Figure 6 This is a schematic diagram of the chip structure provided in an embodiment of the present invention. Detailed Implementation
[0049] To facilitate a clear description of the technical solutions in the embodiments of the present invention, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. For example, the first threshold and the second threshold are merely used to distinguish different thresholds and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" are not necessarily different.
[0050] It should be noted that in this invention, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.
[0051] In this invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a combination of a and b, a combination of a and c, a combination of b and c, or a, b, and c, where a, b, and c can be single or multiple.
[0052] Figure 1 This document illustrates a flowchart of an acceleration method for a fully binary graph convolutional network based on RRAM, as provided in an embodiment of this application. Figure 1 As shown, the acceleration method for the RRAM-based fully binary graph convolutional network includes:
[0053] Step 101: In the binary combination stage of the Lth layer in the binary graph convolutional network, the node features of the graph data are extracted through a fully connected layer with binary weights and binarized to obtain the first binarized node features.
[0054] In this application, during the binary combination stage of the Lth layer in the binary graph convolutional network, the node features of the graph data are extracted through a fully connected layer with binary weights using the Sign function, and then binarized to obtain the first binarized node features.
[0055] Step 102: In the binary aggregation stage of the Lth layer in the binary graph convolutional network, the first binary node features are aggregated and binarized based on the neighboring nodes of each node to obtain the updated node features.
[0056] In this application, during the binary aggregation stage of the Lth binary graph convolutional layer, the neighbor node features of each node and the first binary node features corresponding to the node are added together, the average is taken, and then binarization is performed to obtain the updated node features.
[0057] Step 103: Iterate through the binary combination stage and the binary aggregation stage for all nodes in the node set to obtain the optimized node features.
[0058] Step 104: Classify the graph data or nodes based on the node features.
[0059] The present invention provides an acceleration method for RRAM-based fully binary graph convolutional networks, applied to binary graph convolutional networks. The method includes: in the binary combination stage of the Lth layer of the binary graph convolutional network, extracting node features of graph data through a fully connected layer with binary weights and performing binarization processing to obtain first binary node features; in the binary aggregation stage of the Lth layer of the binary graph convolutional network, aggregating and binarizing the first binary node features based on the neighbor nodes of each node to obtain updated node features; iteratively performing the binary combination stage and the binary aggregation stage on all nodes in the node set to obtain optimized node features; and classifying graph data or nodes based on the node features, which can greatly reduce model parameters and graph data memory while ensuring recognition rate and reducing computational power consumption.
[0060] Optional, Figure 2 This document illustrates a flowchart of another acceleration method for RRAM-based fully binary graph convolutional networks provided in an embodiment of this application. (See attached diagram.) Figure 2 The acceleration method for RRAM-based fully binary graph convolutional networks includes:
[0061] Step 201: In the binary combination stage of the Lth layer in the binary graph convolutional network, the node features of the graph data are extracted through a fully connected layer with binary weights by combining the Sign function, and then binarized to obtain the first binarized node features.
[0062] Graph data is a data structure consisting of a set of nodes and a set of edges representing the relationships between nodes. Graph data can be represented as G(V, E), where V is the set of nodes and E is the set of edges. Node characteristics are... The neighboring nodes of a node are
[0063] In this application, the implementation process of step 201 above may include the following sub-steps:
[0064] Sub-step A1: In the binary combination stage of the Lth layer in the binary graph convolutional network, the full-precision weights are binarized using the Sign function to obtain binarized weights.
[0065] The Sign function can be used to binarize the weights and net liveness values.
[0066]
[0067] Where x represents a variable, x b This represents the binary variable corresponding to the variable.
[0068] Specifically, the full-precision weights W can be binarized using the Sign function to obtain the binarized weights W. b .
[0069] Sub-step A2: For each node, extract the node features from the previous layer using the binary weights through a matrix-vector multiplication operation.
[0070] In this application, for each node, the node features of the (L-1)th layer are extracted by matrix-vector multiplication using binarized weights.
[0071] Sub-step A3: Binarize the node features to obtain the first binarized node features.
[0072] Figure 3 This illustration shows a schematic diagram of a hardware architecture for accelerating a fully binary graph convolutional network based on RRAM, as provided in an embodiment of this application. Figure 3 As shown, the hardware architecture includes a retrieval engine 01, an aggregation engine 02, and a combination engine 03. Specifically, the retrieval engine 01 is composed of an RRAM-based CAM array. The CAM performs parallel search operations by comparing input data with all stored data and returning the addresses of matching stored data. The retrieval engine 01, composed of the CAM array, stores the source node address of each edge. When searching for the neighbor nodes of each source node, the CAM array uses the source node address as input data and retrieves the edges connected to that source node stored in the CAM. If the source node of the edge is the search node, it outputs "1"; otherwise, it outputs "0".
[0073] Step 202: In the binary aggregation stage of the Lth binary graph convolutional layer, the neighbor node features of each node and the first binary node features corresponding to the node are added together, the average value is taken, and then binarization is performed to obtain the updated node features.
[0074] In this application, the output vector of the retrieval engine 01 is used as the input of the aggregation engine and multiplied with the target node features stored in the MAC array to achieve the aggregation of neighbor node features of each source node.
[0075] Step 203: Iterate through the binary combination stage and the binary aggregation stage for all nodes in the node set to obtain the optimized node features.
[0076] In this application, the combination engine 03 is composed of an RRAM-based MAC array, which stores binary weights. The node features updated by the aggregation engine 02 are used as input, and node feature extraction is achieved through binary weights.
[0077] Step 204: Classify the graph data or nodes based on the node features.
[0078] In this application, through multiple iterations of retrieval engine 01, aggregation engine 02 and combination engine 03, the nodes or graphs in the node set are correctly classified.
[0079] Table 1 shows various datasets provided in the embodiments of this application. As shown in Table 1, the datasets may include MUTAG, PTC, Cora, and CiteSeer:
[0080] Table 1
[0081] Dataset Task category picture Average number of nodes Average number of edges MUTAG picture 2 188 17.93 19.79 PTC picture 2 344 14.29 14.69 Dataset Task category node side Node feature dimensions Cora node 7 2708 5429 1433 CiteSeer node 6 3312 4732 3703
[0082] Step 205: In the backpropagation phase, the binary graph convolutional network is corrected using the cross-entropy loss function.
[0083] In this application, the loss function used in the BGCN model during backpropagation is the cross-entropy loss function:
[0084] Among them, y L Let F be the set of labeled nodes, Y be the number of categories, and Z be the predicted value of the L-th layer BGCN. Since the Sign function is not differentiable at zero, its derivative is... Zero, using STE estimation:
[0085] g r =g q 1 |r|≤1Equation (3).
[0086] Table 2 shows a comparison of BGCN and GCN in terms of recognition rate, model, and graph data memory usage provided in the embodiments of this application:
[0087] Table 2
[0088]
[0089] As can be seen from Table 2 in this application, the acceleration method of RRAM-based full binary graph convolutional network in this application embodiment can greatly reduce model parameters and graph data memory while ensuring recognition rate, and at the same time reduce computational power consumption.
[0090] The present invention provides an acceleration method for RRAM-based fully binary graph convolutional networks, applied to binary graph convolutional networks. The method includes: in the binary combination stage of the Lth layer of the binary graph convolutional network, extracting node features of graph data through a fully connected layer with binary weights and performing binarization processing to obtain first binary node features; in the binary aggregation stage of the Lth layer of the binary graph convolutional network, aggregating and binarizing the first binary node features based on the neighbor nodes of each node to obtain updated node features; iteratively performing the binary combination stage and the binary aggregation stage on all nodes in the node set to obtain optimized node features; and classifying graph data or nodes based on the node features, which can greatly reduce model parameters and graph data memory while ensuring recognition rate and reducing computational power consumption.
[0091] Figure 4 This illustration shows a schematic diagram of the structure of an acceleration device for a fully binary graph convolutional network based on RRAM, as provided in an embodiment of this application. Figure 4 As shown, the acceleration device 300 for the RRAM-based fully binary graph convolutional network includes:
[0092] The first binarization processing module 301 is used to extract the node features of the graph data through a fully connected layer with binary weights and perform binarization processing in the binary combination stage of the Lth layer in the binary graph convolutional network to obtain the first binarized node features.
[0093] The second binarization processing module 302 is used to perform aggregation and binarization processing on the first binarized node features based on the neighbor nodes of each node in the binary aggregation stage of the Lth layer of the binary graph convolutional network to obtain the updated node features.
[0094] Iteration module 303 is used to iteratively perform the binary combination stage and the binary aggregation stage on all the nodes in the node set to obtain the optimized node features;
[0095] The classification module 304 is used to classify graph data or nodes based on the node features.
[0096] Optionally, the first binarization processing module includes:
[0097] The first binarization processing submodule is used to extract the node features of the graph data through a fully connected layer with binary weights in the binary combination stage of the Lth layer in the binary graph convolutional network, and perform binarization processing to obtain the first binarized node features.
[0098] The first binarization processing submodule includes:
[0099] The first binarization unit is used to binarize the full-precision weights through the Sign function in the binary combination stage of the Lth layer in the binary graph convolutional network to obtain binarized weights.
[0100] The node feature extraction unit is used to extract node features for each node by using the binarized weights through matrix-vector multiplication operations on the node features of the previous layer.
[0101] The second binarization unit is used to binarize the node features to obtain the first binarized node features.
[0102] Optionally, the second binarization processing module includes:
[0103] The second binarization processing submodule is used in the binary aggregation stage of the Lth layer binary graph convolutional layer to add the neighbor node features of each node and the first binarized node features corresponding to the node, take the average, and then perform binarization processing to obtain the updated node features.
[0104] Optionally, the device further includes:
[0105] A correction module is used to correct the binary graph convolutional network using the cross-entropy loss function during the backpropagation phase.
[0106] The acceleration device for a fully binary graph convolutional network based on RRAM provided in this embodiment of the invention includes: a first binarization processing module, used to extract node features of graph data through a fully connected layer with binary weights and perform binarization processing in the binary combination stage of the Lth layer of the binary graph convolutional network to obtain first binarized node features; a second binarization processing module, used to aggregate and binarize the first binarized node features based on the neighbor nodes of each node in the binary aggregation stage of the Lth layer of the binary graph convolutional network to obtain updated node features; an iteration module, used to iteratively perform the binary combination stage and the binary aggregation stage on all nodes in the node set to obtain optimized node features; and a classification module, used to classify graph data or nodes based on the node features, which can greatly reduce model parameters and graph data memory while ensuring recognition rate and reducing computational power consumption.
[0107] This invention provides an acceleration device for RRAM-based fully binary graph convolutional networks, applicable to, for example... Figures 1 to 2 To avoid repetition, the acceleration methods for any of the RRAM-based full binary graph convolutional networks shown will not be elaborated here.
[0108] The electronic device in this embodiment of the invention can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, a mobile electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc., while a non-mobile electronic device can be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This embodiment of the invention does not impose specific limitations.
[0109] The electronic device in this embodiment of the invention can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment of the invention does not impose specific limitations.
[0110] Figure 5 A schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention is shown. Figure 5 As shown, the electronic device 400 includes a processor 410.
[0111] like Figure 5 As shown, the processor 410 described above can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention.
[0112] like Figure 5 As shown, the electronic device 400 may further include a communication line 440. The communication line 440 may include a path for transmitting information between the components.
[0113] Optional, such as Figure 5 As shown, the above-described electronic device may further include a communication interface 420. There may be one or more communication interfaces 420. The communication interface 420 may use any transceiver-like device for communicating with other devices or communication networks.
[0114] Optional, such as Figure 5 As shown, the electronic device may further include a memory 430. The memory 430 stores computer execution instructions for implementing the present invention, and its execution is controlled by a processor. The processor executes the computer execution instructions stored in the memory to implement the method provided in the embodiments of the present invention.
[0115] like Figure 5 As shown, memory 430 can be read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 430 can exist independently and be connected to processor 410 via communication line 440. Memory 430 can also be integrated with processor 410.
[0116] Optionally, the computer execution instructions in the embodiments of the present invention may also be referred to as application code, and the embodiments of the present invention do not specifically limit this.
[0117] In a specific implementation, as one example, such as Figure 5 As shown, processor 410 may include one or more CPUs, such as Figure 5 CPU0 and CPU1 in the CPU.
[0118] In a specific implementation, as one example, such as Figure 5 As shown, the terminal device may include multiple processors, such as Figure 5 The first processor 4101 and the second processor 4102 are included. Each of these processors can be a single-core processor or a multi-core processor.
[0119] Figure 6 This is a schematic diagram of the chip structure provided in an embodiment of the present invention. Figure 6 As shown, the chip 500 includes one or more processors 410.
[0120] Optional, such as Figure 6 As shown, the chip also includes a communication interface 420 and a memory 430. The memory 430 may include read-only memory and random access memory, and provides operation instructions and data to the processor. A portion of the memory may also include non-volatile random access memory (NVRAM).
[0121] In some implementations, such as Figure 6 As shown, memory 430 stores the following elements: execution modules or data structures, or subsets thereof, or extended sets thereof.
[0122] In embodiments of the present invention, such as Figure 6 As shown, the corresponding operation is executed by calling the operation instructions stored in the memory (which can be stored in the operating system).
[0123] like Figure 6 As shown, the processor 410 controls the processing operations of any one of the terminal devices. The processor 410 can also be called a central processing unit (CPU).
[0124] like Figure 6As shown, memory 430 may include read-only memory and random access memory, providing instructions and data to the processor. A portion of memory 430 may also include NVRAM. For example, in an application, memory, communication interfaces, and memory are coupled together via a bus system, which may include, in addition to a data bus, a power bus, a control bus, and a status signal bus, etc. However, for clarity, in... Figure 6 The general labeled all buses as Bus System 540.
[0125] like Figure 6 As shown, the methods disclosed in the above embodiments of the present invention can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.
[0126] On the one hand, a computer-readable storage medium is provided, which stores instructions that, when executed, implement the functions performed by the terminal device in the above embodiments.
[0127] On the one hand, a chip is provided for use in a terminal device. The chip includes at least one processor and a communication interface, the communication interface and the at least one processor being coupled together. The processor is used to run instructions to implement the functions performed by the accelerated method of the RRAM-based full binary graph convolutional network in the above embodiments.
[0128] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are performed entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a terminal, a user equipment, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD).
[0129] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, the disclosure, and the appended claims in carrying out the claimed invention. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.
[0130] Although the invention has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made therein without departing from the spirit and scope of the invention. Accordingly, this specification and drawings are merely exemplary descriptions of the invention as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if such modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include such modifications and modifications.
Claims
1. An acceleration method for fully binary graph convolutional networks based on RRAM, characterized in that, The method is applied in a hardware architecture, which includes a retrieval engine, an aggregation engine, and a combination engine. The retrieval engine is composed of a content-addressable memory (CAM) array based on resistive random access memory (RRAM), and the combination engine is composed of a multiply-accumulate MAC array based on RRAM. In the binary combination stage of the Lth layer in the binary graph convolutional network, the combination engine extracts node features from the graph data based on the binary weights stored in the MAC array and performs binarization processing to obtain the first binarized node features. In the binary aggregation stage of the Lth layer in the binary graph convolutional network, the retrieval engine retrieves the neighboring nodes of each node, and uses the output vector of the retrieval engine as the input of the aggregation engine. This vector is multiplied with the target node features stored in the MAC array to achieve neighbor node feature aggregation and binarization to obtain the updated node features. The binary combination stage and the binary aggregation stage are iteratively performed on all nodes in the node set to obtain the optimized node features; The graph data or nodes are classified based on the node features. In the binary aggregation stage of the Lth layer of the binary graph convolutional network, the first binarized node features are aggregated and binarized based on the neighboring nodes of each node to obtain updated node features, including: In the binary aggregation stage of the Lth binary graph convolutional layer, the neighbor node features of each node and the first binary node features corresponding to the node are added together, the average is taken, and then binarization is performed to obtain the updated node features.
2. The method according to claim 1, characterized in that, In the binary combination stage of the Lth layer in the binary graph convolutional network, the combination engine extracts node features from the graph data based on the binary weights stored in the MAC array, performs binarization processing, and obtains the first binarized node features, including: In the binary combination stage of the Lth layer in the binary graph convolutional network, the combination engine extracts node features from the graph data based on the binary weights stored in the MAC array and combines them with the Sign function, and performs binarization processing to obtain the first binary node features.
3. The method according to claim 2, characterized in that, In the binary combination stage of the Lth layer in the binary graph convolutional network, the combination engine extracts node features from the graph data based on the binary weights stored in the MAC array and combines them with the Sign function, and performs binarization processing to obtain the first binary node features, including: In the binary combination stage of the Lth layer in the binary graph convolutional network, the full-precision weights are binarized by the Sign function to obtain the binarized weights. For each node, the node features of the previous layer are extracted by matrix-vector multiplication using the binarized weights; The node features are binarized to obtain the first binarized node features.
4. The method according to claim 1, characterized in that, After classifying graph data or nodes based on the node features, the method further includes: During the backpropagation phase, the binary graph convolutional network is corrected using the cross-entropy loss function.
5. An acceleration device for a fully binary graph convolutional network based on RRAM, characterized in that, The device is applied in a hardware architecture that includes a retrieval engine, an aggregation engine, and a combination engine. The retrieval engine is composed of a content-addressable memory (CAM) array based on resistive random access memory (RRAM), and the combination engine is composed of a multiply-accumulate MAC array based on RRAM. The device includes: The first binarization processing module is used to control the combination engine to extract node features from the graph data and perform binarization processing on the node features of the graph data based on the binary weights stored in the MAC array during the binary combination stage of the Lth layer in the binary graph convolutional network, so as to obtain the first binarized node features. The second binarization processing module is used to control the retrieval engine and the aggregation engine. In the binary aggregation stage of the Lth layer in the binary graph convolutional network, the retrieval engine retrieves the neighboring nodes of each node, and uses the output vector of the retrieval engine as the input of the aggregation engine. It is multiplied with the target node features stored in the MAC array to realize the aggregation of neighboring node features and perform binarization processing to obtain the updated node features. The iteration module is used to iteratively perform the binary combination stage and the binary aggregation stage on all the nodes in the node set to obtain the optimized node features; A classification module is used to classify graph data or nodes based on the node features; The second binarization processing module includes: The second binarization processing submodule is used in the binary aggregation stage of the Lth layer binary graph convolutional layer to add the neighbor node features of each node and the first binarized node features corresponding to the node, take the average value, and then perform binarization processing to obtain the updated node features.
6. The apparatus according to claim 5, characterized in that, The first binarization processing module includes: The first binarization processing submodule is used to control the combination engine. In the binary combination stage of the Lth layer in the binary graph convolutional network, based on the binary weights stored in the MAC array, the node features of the graph data are extracted by combining the Sign function and binarized to obtain the first binarized node features. The first binarization processing submodule includes: The first binarization unit is used to binarize the full-precision weights through the Sign function in the binary combination stage of the Lth layer in the binary graph convolutional network to obtain binarized weights. The node feature extraction unit is used to extract node features for each node by using the binarized weights through matrix-vector multiplication operations on the node features of the previous layer. The second binarization unit is used to binarize the node features to obtain the first binarized node features.
7. The apparatus according to claim 5, characterized in that, The device further includes: A correction module is used to correct the binary graph convolutional network using the cross-entropy loss function during the backpropagation phase.
8. An electronic device, characterized in that, include: One or more processors; And one or more machine-readable media thereon storing instructions, which, when executed by the one or more processors, cause the accelerated method of the RRAM-based fully binary graph convolutional network of any one of claims 1-4 to be performed.