Sparse matrix continuous multiplication method and device

Abstract

The invention provides a sparse matrix multiplication method and device, and belongs to the technical field of computers. According to the sparse matrix multiplication method provided by the invention, in the process of calculating the sparseness of the intermediate result matrix, calculation is carried out through the row sparseness of the plurality of sparse matrixes in the sparse matrix sequence, and compared with the whole sparseness of the sparse matrixes, the granularity of the row sparseness is smaller, so that the sparseness of the intermediate result matrix is improved. The accuracy of the sparseness of the intermediate result matrix can be higher. Sparse matrix continuous multiplication is carried out according to the sparsity with relatively high accuracy, so that the efficiency of sparse matrix continuous multiplication can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a sparse matrix multiplication method and device. Background Art

[0002] A sparse matrix is a matrix whose elements are mostly zero. With the development of technologies such as artificial intelligence, virtual reality, and high-performance computing, sparse matrix multiplication has been widely used in fields such as e-commerce, social networks, and biology. In any field, interactions between different objects are often represented using sparse matrices. For example, in e-commerce, a user's purchase history is often modeled as a large sparse matrix. That is, if the mth user purchased the kth item, then the element in the mth row and kth column of the matrix is 1. m and k are both integers greater than zero.

[0003] Currently, a sparse matrix multiplication method is to use the ratio of the number of non-zero elements in the sparse matrix as the sparsity of the sparse matrix, and the result of the multiplication of multiple sparse matrices of a subsequence in a sparse matrix sequence is called an intermediate result matrix. During the multiplication process, the sparsity of the sparse matrix is used as the probability that each element in the sparse matrix is non-zero. For each subsequence, based on the probability that each element in the multiple sparse matrices in the subsequence is non-zero, the sparsity of the intermediate result matrix corresponding to the subsequence is estimated to obtain the sparsity of multiple intermediate result matrices. Based on the sparsity of the multiple intermediate result matrices, the multiplication order of the multiple sparse matrices in the sparse matrix sequence is determined, and the sparse matrix multiplication operation is performed according to the multiplication order.

[0004] Since in the sparse matrix multiplication calculation, the elements in the intermediate result matrix are obtained by multiplying the corresponding rows and columns in the sparse matrix, and have a low correlation with the entire sparse matrix, estimating the sparsity of the intermediate result matrix based on the overall sparsity of the sparse matrix has low accuracy. Performing sparse matrix multiplication based on the less accurate sparsity will result in low efficiency of sparse matrix multiplication. Summary of the Invention

[0005] The embodiments of the present application provide a sparse matrix multiplication method and apparatus, which can improve the efficiency of sparse matrix multiplication. The technical solution is as follows.

[0006] In a first aspect, a sparse matrix multiplication method is provided, the method comprising: obtaining a sparse matrix sequence, the sparse matrix sequence comprising multiple sparse matrices to be multiplied, the elements in each sparse matrix being used to represent the interactive relationship between nodes; determining the sparsity of intermediate result matrices of multiple subsequences in the sparse matrix sequence based on the row sparsity of the multiple sparse matrices, each subsequence comprising at least two adjacent sparse matrices in the sparse matrix sequence, the intermediate result matrix referring to the multiplication result of the sparse matrices in the subsequence; determining the multiplication order of the multiple sparse matrices in the sparse matrix sequence based on the sparsity of the multiple intermediate result matrices; and performing multiplication operations on the multiple sparse matrices in the sparse matrix sequence based on the multiplication order.

[0007] Among them, a sparse matrix is a sparse matrix in any field. For example, when a sparse matrix is used in the field of e-commerce, if the nodes represent users, goods, and suppliers, the elements in the sparse matrix are used to represent the purchase relationship between users and goods, the supply relationship between goods and suppliers, etc. Taking a sparse matrix representing a purchase relationship as an example, in the sparse matrix, the element located in the mth row and the kth column is 1, which means that the mth user purchased the kth item, and the element located in the mth row and the kth column is 0, which means that the mth user did not purchase the kth item. When a sparse matrix is used in the field of social networks, if the nodes represent different users, the elements in the sparse matrix are used to represent the interaction between different users, which includes likes, forwarding, and sending messages. When a sparse matrix is used in the field of biology, if the nodes represent different molecules, the elements in the sparse matrix are used to represent the chemical reactions between different molecules. If the nodes represent patients, drugs, and drug reactions, the elements in the sparse matrix are used to represent the patient's medication status and the efficacy of the drug, etc. The embodiments of this application do not limit the application scenarios of sparse matrices. Sparsity refers to the ratio of the number of non-zero elements to all elements.

[0008] The sparse matrix multiplication method provided in the embodiments of the present application calculates the sparsity of an intermediate result matrix by calculating the row sparsity of multiple sparse matrices in a sparse matrix sequence. Compared with using the sparsity of the sparse matrix as a whole, the row sparsity has a smaller granularity, which can make the sparsity of the intermediate result matrix more accurate. Performing sparse matrix multiplication based on the more accurate sparsity can improve the efficiency of sparse matrix multiplication.

[0009] In some embodiments, the above-mentioned determination of the sparsity of the intermediate result matrices of multiple subsequences in the sparse matrix sequence based on the row sparsity of multiple sparse matrices includes: for any subsequence, based on the row sparsity of the last sparse matrix in the subsequence and the elements in the sparse matrices other than the last sparse matrix, obtaining the row sparsity of the intermediate result matrix of the subsequence; and obtaining the sparsity of the intermediate result matrix of the subsequence based on the row sparsity of the intermediate result matrix of the subsequence. The above-mentioned process first obtains the row sparsity of the intermediate result matrix row by row, and then integrates the row sparsity into the sparsity of the entire intermediate result matrix. Compared with directly obtaining the sparsity of the entire intermediate result matrix, the granularity of the row sparsity is smaller, which can make the row sparsity of the intermediate result matrix more accurate, thereby improving the accuracy of the sparsity of the intermediate result matrix.

[0010] In some embodiments, the above-mentioned process of obtaining the row sparsity of the intermediate result matrix of the subsequence based on the row sparsity of the last sparse matrix in the subsequence and the elements in the sparse matrices other than the last sparse matrix includes multiple rounds of calculation processes, and the multiple rounds of calculation processes include: taking the subsequence as an example including p sparse matrices, in the first round of calculation, based on the row sparsity of the p-th sparse matrix in the subsequence and the elements in the p-1-th sparse matrix in the subsequence, the row sparsity of the first round is calculated, where p is an integer greater than or equal to 2; in the i-th round of calculation, based on the row sparsity of the i-1-th round and the elements in the pi-th sparse matrix in the subsequence, the row sparsity of the i-th round is calculated, where i is an integer greater than or equal to 2. The above process calculates the row sparsity in order from right to left, so that in the multiple rounds of calculation, the row sparsity of the intermediate result matrix can be obtained without obtaining the position information (structural information) of the non-zero elements in the intermediate result matrix, and the efficiency of obtaining the row sparsity is high.

[0011] In some embodiments, the above-mentioned calculation of the row sparsity of the i-th round based on the row sparsity of the i-1th round and the elements in the pi-th sparse matrix in the subsequence includes: if the row sparsity of the m-th row of the pi-th sparse matrix is 0, then the row sparsity of the m-th row of the i-th round is 0, and m is an integer greater than 0; if the row sparsity of the m-th row of the pi-th sparse matrix is not 0, then based on the elements in the m-th row of the pi-th sparse matrix and the row sparsity of the i-1th round, the row sparsity of the m-th row of the i-th round is calculated. The above process uses the row sparsity of the i-1th round and the elements in the pi-th sparse matrix, which are more relevant to the row sparsity of the i-th round, to calculate the row sparsity of the i-th round row by row, and the accuracy of the calculated row sparsity is higher.

[0012] In some embodiments, the above-mentioned determination of the multiplication order of multiple sparse matrices in the sparse matrix sequence based on the sparsity of multiple intermediate result matrices includes: for any subsequence, based on the sparsity of multiple intermediate result matrices, obtaining the multiplication time of the subsequence, and obtaining the multiplication time of multiple subsequences; based on the multiplication time of multiple subsequences, determining the multiplication order of multiple sparse matrices in the sparse matrix sequence. The above process determines the multiplication order of the sparse matrix sequence based on the multiplication time of the subsequence, and can relatively quickly and accurately find the multiplication order with the shortest multiplication time, making the sparse matrix multiplication more efficient.

[0013] In some embodiments, the above-mentioned obtaining of the continuous multiplication time of the subsequence based on the sparsity of multiple intermediate result matrices includes: for any first subsequence, based on the sparsity of the sparse matrix in the first subsequence and the sparsity of the intermediate result matrix of the first subsequence, obtaining the continuous multiplication time of the first subsequence, where the first subsequence is a subsequence containing two sparse matrices; for any second subsequence, dividing the second subsequence according to different division methods to obtain subsequence combinations under different division methods, each subsequence combination includes a front subsequence and a back subsequence, the order of the front subsequence in the second subsequence is before the back subsequence, and the second subsequence is a subsequence containing at least three sparse matrices; for any subsequence combination, based on the sparsity of the intermediate result matrix of the front subsequence, the sparsity of the intermediate result matrix of the back subsequence, and the sparsity of the intermediate result matrix of the second subsequence in the subsequence combination, obtaining the continuous multiplication time of the subsequence combination; and taking the shortest continuous multiplication time among the multiple subsequence combinations as the continuous multiplication time of the second subsequence. The above process obtains the shortest multiplication time of each subsequence, and determines the multiplication order according to the shortest multiplication time, which can make the multiplication order more accurate.

[0014] In some embodiments, the above-mentioned determination of the order of multiplication of multiple sparse matrices in the sparse matrix sequence based on the multiplication time of multiple subsequences includes multiple rounds of division processes, and the multiple rounds of division processes include: in the first round of division, the sparse matrix sequence is divided into two subsequences, and the subsequence with the shorter multiplication time of the two subsequences is determined as the subsequence on which the multiplication operation is performed first, and the two subsequences corresponding to the division method are the subsequences that make the multiplication time of the sparse matrix sequence the shortest; in the nth round of division, for any subsequence divided in the n-1th round of division, the subsequence is divided into a third subsequence and a fourth subsequence, and the third subsequence and the fourth subsequence corresponding to the division method are the subsequences that make the multiplication time of the subsequences the shortest, and n is an integer greater than or equal to 2; the subsequence with the shorter multiplication time of the third subsequence and the fourth subsequence is determined as the subsequence on which the multiplication operation is performed first. The above process divides the subsequences and then determines the multiplication order according to the multiplication times of the subsequences. This process does not require calculating the multiplication times for each multiplication order one by one, and thus the multiplication order can be obtained more quickly.

[0015] In some embodiments, the above-mentioned determination of the multiplication order of multiple sparse matrices in the sparse matrix sequence based on the multiplication time of multiple subsequences includes: obtaining the multiplication time of multiple candidate multiplication orders of the sparse matrix sequence based on the multiplication time of multiple subsequences, the candidate multiplication order including a combination of multiple non-overlapping subsequences; and determining the candidate multiplication order with the shortest multiplication time among the multiple candidate multiplication orders as the multiplication order of the multiple sparse matrices in the sparse matrix sequence. The above process first obtains all possible candidate multiplication orders, and then determines the candidate multiplication order with the shortest multiplication time as the multiplication order of the sparse matrix sequence, and the accuracy of the multiplication order is high.

[0016] In some embodiments, the non-zero elements of each row in the sparse matrix are stored in different arrays, so that in the multiplication calculation of the sparse matrix, the non-zero elements of different rows can be read in parallel, thereby enabling the multiplication calculation of different rows to be performed in parallel. The multiplication result of the two sparse matrices is also stored in a manner that the non-zero elements of each row are stored in different arrays, so that the storage of the multiplication results of different rows can be parallel. In other words, the multiplication result of the current row does not need to wait until the multiplication results of the previous row are fully stored before being stored, which can improve the parallel efficiency of the multiplication calculation of different rows of the sparse matrix, thereby improving the efficiency of the sparse matrix multiplication.

[0017] In a second aspect, a sparse matrix multiplication device is provided for executing the above-mentioned sparse matrix multiplication method. Specifically, the sparse matrix multiplication device includes a functional module for executing the sparse matrix multiplication method provided in the above-mentioned first aspect or any optional manner of the above-mentioned first aspect.

[0018] In a third aspect, a computing device or a computing device cluster is provided, which includes a processor for executing program code so that the computing device or the computing device cluster executes to implement operations performed by the above-mentioned sparse matrix multiplication method.

[0019] In a fourth aspect, a computer-readable storage medium is provided, in which at least one program code is stored. The program code is read by a processor to enable a computing device to perform operations such as those performed by the above-mentioned sparse matrix multiplication method.

[0020] In a fifth aspect, a computer program product or computer program is provided, which includes a program code, and the program code is stored in a computer-readable storage medium. The processor of a computing device reads the program code from the computer-readable storage medium, and the processor executes the program code, so that the computing device executes the method provided in the first aspect or any optional embodiment of the first aspect.

[0021] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a structural diagram of a computing device provided in an embodiment of the present application;

[0023] Figure 2 This is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0024] Figure 3 This is a flowchart of a sparse matrix multiplication method provided in an embodiment of the present application;

[0025] Figure 4 This is an example diagram of a matrix storage method provided in an embodiment of the present application;

[0026] Figure 5 This is a flowchart of a sparse matrix row sparsity calculation provided by an embodiment of the present application;

[0027] Figure 6 This is an example diagram of sparse matrix row sparsity calculation provided by an embodiment of the present application;

[0028] Figure 7 This is a flowchart of calculating the sparsity of an intermediate result matrix of multiple subsequences provided in an embodiment of the present application;

[0029] Figure 8 This is a flowchart of a software test code generation task processing provided by an embodiment of the present application;

[0030] Figure 9 This is an example diagram of node embedding provided by an embodiment of the present application;

[0031] Figure 10 It is a structural diagram of a sparse matrix multiplication device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0033] Figure 1 1 is a schematic diagram of a computing device provided in an embodiment of the present application. It should be understood that the computing device described below can implement any function in any of the following methods. Generally, the computing device 100 includes: a processor 101 and a memory 102.

[0034] The processor 101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 101 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 101 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 101 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 101 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0035] The memory 102 may include one or more computer-readable storage media, which may be non-transitory. The memory 102 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 102 is used to store at least one program, which is used to be executed by the processor 101 to implement the sparse matrix multiplication method provided in the method embodiment of the present application.

[0036] In some embodiments, computing device 100 may optionally include a peripheral device interface 103 and at least one peripheral device. Processor 101, memory 102, and peripheral device interface 103 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 103 via a bus, signal lines, or circuit boards.

[0037] In the embodiment of the present application, the computing device 100 can be configured as a terminal or a server, and the terminal or the server serves as the execution entity to implement the technical solution provided in the embodiment of the present application.

[0038] In some embodiments, computing device 100 may be a portable mobile terminal, such as a smartphone, a tablet computer, a Moving Picture Experts Group Audio Layer III (MP3) player, a Moving Picture Experts Group Audio Layer IV (MP4) player, a laptop computer, or a desktop computer. Computing device 100 may also be referred to as a user device, a portable terminal, a laptop terminal, a desktop terminal, or other names.

[0039] In some embodiments, the computing device 100 may be a separate physical server, or may be implemented as Figure 2 The computing device cluster shown is a server cluster or distributed file system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Taking the cloud server as an example, the computing device can also be referred to as a cloud platform (short for cloud computing platform), which refers to services based on hardware and software resources and provides computing, network, and storage capabilities. Through the network "cloud," massive amounts of data are processed and analyzed remotely and then returned to users. This platform features large-scale, distributed, virtualized, highly available, scalable, on-demand services, and security. Cloud platforms enable the rapid provisioning and release of configurable computing resources with minimal management overhead and low interaction complexity between users and service providers.

[0040] The sparse matrix multiplication calculation described above is applied in various fields, such as e-commerce, social networking, data mining, information retrieval, and biology. In any field, knowledge graphs are used to organize and represent information. Specifically, multiple nodes and their interactions are used to represent the entities and interactions between them within that field. In a knowledge graph, each node represents an entity, and the interactions between different nodes are represented by elements in a sparse matrix. During the construction of a knowledge graph, a knowledge graph node embedding algorithm is employed to enrich the information within the knowledge graph. Specifically, sparse matrix multiplication is used to integrate the interactions between different nodes, thereby deriving unknown interactions between nodes based on known interactions between them. When a new node is generated, a node matching the new node's attribute information can be retrieved from the knowledge graph based on the new node's attribute information. Based on the interactions between the new node and other nodes, the interactions between the new node and other nodes can be predicted. These interactions between nodes can be used to implement tasks such as intelligent recommendation, data mining, and search engine optimization.

[0041] For example, when applying knowledge graphs to e-commerce, a first sparse matrix is used to represent the known purchasing relationships between users and products, and a second sparse matrix is used to represent the known supply relationships between products and suppliers. By multiplying the first and second sparse matrices, the resulting matrix can be used to infer the purchasing relationship between the user and the supplier. When a user browses a shopping page, products from suppliers frequently purchased by the user can be recommended based on this purchasing relationship. When applying knowledge graphs to social networks, different sparse matrices are used to represent known interactions between users with different characteristics. By multiplying multiple sparse matrices, the resulting matrix can be used to infer potential interactions between users with different characteristics. Based on this potential interaction, users that may be of interest to the user can be recommended. When applying knowledge graphs to biology, multiple sparse matrices are used to represent the relationships between patients, drugs, and drug reactions. By multiplying multiple sparse matrices, the knowledge graph can acquire more complex medical knowledge to assist in clinical decision-making.

[0042] In addition to the aforementioned knowledge graphs, sparse matrix multiplication is also widely used in fields such as computer vision, natural language processing, and machine learning. Sparse matrices are used to represent the input data and parameters of the model in any of these fields, and the output data of the model can be obtained through sparse matrix multiplication. Taking natural language processing as an example, the natural language processing task calls a natural language processing model to process the input data. The natural language model includes multiple neural network layers, and sparse matrices are used to represent the parameters of each neural network layer. During the processing, the text and the relationships between texts are converted into sparse matrices, which are used as the input data of the natural language processing model. The sparse matrix representing the input data is multiplied with the sparse matrices representing the parameters of each neural network layer to obtain the output data of the natural language processing model.

[0043] In order to increase the speed of sparse matrix multiplication calculations and improve the efficiency of sparse matrix multiplication, an embodiment of the present application provides a sparse matrix multiplication method: in the process of calculating the sparsity of an intermediate result matrix, the calculation is performed using the row sparsity of multiple sparse matrices in a sparse matrix sequence. Compared with using the sparsity of the sparse matrix as a whole, the granularity of the row sparsity is smaller, which can make the sparsity of the intermediate result matrix more accurate. Performing sparse matrix multiplication based on the more accurate sparsity can improve the efficiency of sparse matrix multiplication.

[0044] The following describes in detail the sparse matrix multiplication method provided in the embodiment of the present application. Figure 3 : This is a flowchart of a sparse matrix multiplication method provided according to an embodiment of the present application. Taking a computing device executing the above sparse matrix multiplication method as an example, the method includes the following steps:

[0045] 301. A computing device obtains a sparse matrix sequence, where the sparse matrix sequence includes multiple sparse matrices to be multiplied, and elements in each sparse matrix are used to represent an interactive relationship between nodes.

[0046] The sparse matrix sequence is a set of sparse matrices arranged in a preset order, and the sparse matrix sequence includes at least two sparse matrices. The preset order is the default multiplication order of sparse matrices. i Represents the i-th sparse matrix in a sparse matrix sequence. If a sparse matrix sequence is {A1, A2, A3, A4, A5, A6}, then the default multiplication order of the sparse matrix is: (((((A1×A2)×A3)×A4)×A5)×A6).

[0047] In an embodiment of the present application, the computing device can obtain a sparse matrix sequence from different data sources. Accordingly, the process of the computing device obtaining the sparse matrix sequence includes: the computing device receives the sparse matrix sequence sent by other computing devices, or the computing device obtains a pre-stored sparse matrix sequence locally, or the computing device obtains the sparse matrix sequence from a database, which is not limited in this embodiment of the present application.

[0048] In the process of obtaining a sparse matrix sequence, the computing device can obtain the sparse matrix sequence in any of the following ways: one way is for the computing device to directly obtain the complete sparse matrix sequence; another way is for the computing device to obtain sparse matrices one by one and combine multiple sparse matrices into a sparse matrix sequence. For example, in the process of obtaining a sparse matrix sequence through a knowledge graph, there is a sparse matrix between every two adjacent nodes in the knowledge graph, which is used to represent the interactive relationship between the two nodes. The computing device performs a random walk along a preset meta-path to obtain sparse matrices one by one to reflect the relationship between adjacent nodes, wherein the preset meta-path is a preset node type sequence that includes the type of nodes to be obtained at each step of the random walk. During the random walk, the computing device randomly obtains a node of the same type in the knowledge graph based on the type of the first node in the preset meta-path. For each node obtained, the computing device randomly obtains a node of the same type from multiple adjacent nodes of the node based on the type of the next node indicated by the preset meta-path, and adds the sparse matrix between the two nodes to the sparse matrix sequence. The embodiment of the present application does not limit the method of obtaining the sparse matrix sequence.

[0049] In the process of obtaining a sparse matrix, the computing device can directly obtain the sparse matrix, and the computing device can also obtain the data to be processed and generate a sparse matrix corresponding to the data. For example, in the field of e-commerce, the computing device obtains the purchase records of multiple users, extracts the data in the purchase records, and integrates the data into a sparse matrix. The embodiments of the present application are not limited to this.

[0050] In some embodiments, after obtaining the sparse matrix sequence, the computing device stores the non-zero elements of each sparse matrix in the sparse matrix sequence and the position information of the non-zero elements in the sparse matrix to reduce space costs.

[0051] The current storage method for storing non-zero elements in sparse matrices is a sparse matrix row compression structure. For each sparse matrix with a rows and b columns, a sparse matrix row compression structure is used to store the sparse matrix, where a and b are both integers greater than zero. The sparse matrix row compression structure includes a list el and an array row. List el is used to store the non-zero elements in the sparse matrix row by row. Each element in list el is a (col, val) tuple, and each tuple stores a non-zero element. Among them, col is used to store the column of the non-zero element in the sparse matrix, and val is used to store the value of the non-zero element. Array row is used to store the position of the first non-zero element in each row of the sparse matrix in list el. For example, row[h] is used to store the position of the first non-zero element in the hth row of the sparse matrix in list el, where h is an integer greater than zero. Although the above sparse matrix row compression structure can reduce space costs, the above storage method can only start storing the non-zero elements of the current row after all the non-zero elements of the previous row have been stored. That is, in sparse matrix multiplication calculations, the calculation results of the current row can only be stored after all the calculation results of the previous row are stored. The parallel efficiency of multiplication calculations of different rows of the sparse matrix is low, and sparse matrix multiplication includes multiple multiplication calculations, resulting in low efficiency of sparse matrix multiplication.

[0052] In order to optimize the process of obtaining the sparsity of the intermediate result matrix and improve the accuracy of sparsity, the storage method of the sparse matrix is also optimized to improve the parallel efficiency of the multiplication calculation of different rows. In some embodiments, for each sparse matrix with a row and b columns, a sparse adjacent list is used to store the pointers of the various arrays of the sparse matrix. The sparse adjacent list list includes a elements, and each element list[h] stores a pointer to the array arr[h]. The array arr[h] is used to store the non-zero elements in the h-th row of the sparse matrix. In the array arr[h], each non-zero element is stored with a tuple (col, val), col is used to store the column of the non-zero element in the sparse matrix, and val is used to store the value of the non-zero element. Using the above-mentioned sparse adjacent list to store the sparse matrix can make the non-zero elements of each row in the sparse matrix be stored in different arrays, thereby improving the parallel efficiency of the multiplication calculation of different rows of the sparse matrix, effectively utilizing the computing power of multiple CPUs, and improving the efficiency of sparse matrix multiplication.

[0053] The following combination Figure 4 The examples in the above two storage methods are illustrated. Figure 4 This is an example diagram of a matrix storage method provided in an embodiment of the present application. Figure 4 Figure (1) is the matrix to be stored Figure 4 Figure (2) shows the matrix A stored using the sparse matrix row compression structure. Figure 4Figure (3) shows the matrix A stored using a sparse adjacent list. Figure 4 In Figure (3), the three tuples in el store the three non-zero elements in matrix A respectively. The "1" in the first position of the array row indicates that the first non-zero element of the first row of the sparse matrix is stored in the first position of el, and the "3" in the second position of the array row indicates that the first non-zero element of the second row of the sparse matrix is stored in the third position of el. It can be seen that the first two tuples in el store the non-zero elements in the first row of matrix A, and the third tuple stores the elements in the second row of matrix A. The first tuple (1, 1) indicates that the non-zero element in the first column is 1, that is, the element in the first row and first column of the sparse matrix is 1. The second tuple (2, 3) indicates that the non-zero element in the second column is 3, that is, the element in the first row and second column of the sparse matrix is 3. The third tuple (2, 2) indicates that the non-zero element in the second column is 2, that is, the element in the second row and second column of the sparse matrix is 2. Figure 4 In Figure (3), list includes two elements: A[1, *] and A[2, *], which are pointers to array arr[1] storing the non-zero elements of the first row of the sparse matrix and array arr[2] storing the non-zero elements of the second row of the sparse matrix, respectively. arr[1] includes two tuples (1, 1) and (2, 3), and arr[2] includes a tuple (2, 2). The meanings of these tuples are the same as those of the tuples in the sparse matrix row compression structure described above, and are not further described in detail in this embodiment of the present application.

[0054] 302. The computing device divides the sparse matrix sequence to obtain multiple subsequences, each subsequence including at least two adjacent sparse matrices in the sparse matrix sequence.

[0055] The order of the multiple sparse matrices in each subsequence is consistent with the order of the multiple sparse matrices in the above-mentioned preset order, and there may be a relationship of inclusion or being included between the divided subsequences. For example, for the sparse matrix sequence {A1, A2, A3, A4, A5, A6}, the subsequences of the sparse matrix sequence include the subsequence {A1, A2, A3, A4} and the subsequence {A1, A2}, etc., wherein the subsequence {A1, A2, A3, A4} contains the subsequence {A1, A2}.

[0056] In the embodiment of the present application, the computing device can divide the sparse matrix sequence in different ways:

[0057] In some embodiments, the partitioning method includes: the computing device starts from the second-to-last sparse matrix in the sparse matrix sequence, and obtains, from right to left, a subsequence consisting of the sparse matrix and the sparse matrix following the sparse matrix. Taking the sparse matrix sequence {A1, A2, A3, A4, A5, A6} as an example, first for A5, the computing device obtains the subsequence {A5, A6}, then for A4, the computing device obtains the subsequence {A4, A5, A6} and the subsequence {A4, A5}, and so on, to obtain all subsequences of the sparse matrix sequence.

[0058] In other embodiments, the partitioning method includes: the computing device first obtains all subsequences of the sparse matrix sequence that contain two sparse matrices, then obtains all subsequences of the sparse matrix sequence that contain three sparse matrices, and so on, to obtain all subsequences of the sparse matrix sequence. In addition to the above two partitioning methods, there are many other partitioning methods, and the embodiments of the present application do not limit such partitioning methods.

[0059] 303. For the p sparse matrices in any subsequence of the multiple subsequences, the computing device performs multiple rounds of calculations in order from right to left to obtain the row sparsity of the intermediate result matrix of the subsequence. In the first round of calculation, the computing device calculates the row sparsity of the first round based on the row sparsity of the p-th sparse matrix in the subsequence and the elements in the p-1-th sparse matrix in the subsequence, where p is an integer greater than or equal to 2.

[0060] Among them, the intermediate result matrix is the result of the continuous multiplication of multiple sparse matrices in the subsequence. During the above multiple rounds of calculation, the row sparsity of each round is the row sparsity of the result matrix of the intermediate subsequence of that round. The intermediate subsequence is a sequence composed of multiple sparse matrices involved in the calculation in that round. The result matrix of the intermediate subsequence is the result of the continuous multiplication of multiple sparse matrices in the intermediate subsequence. Taking the subsequence {A1, A2, A3, A4, A5, A6} as an example, in the first round of calculation, sparse matrices A5 and A6 are involved, and the intermediate subsequence is {A5, A6}. In the second round of calculation, sparse matrices A4, A5, and A6 are involved, and the intermediate subsequence is {A4, A5, A6}. And so on. After five rounds of calculation, the row sparsity of the intermediate result matrix of the subsequence is obtained.

[0061] In step 303, the intermediate subsequence of the first round includes the p-1th sparse matrix and the pth sparse matrix in the subsequence, that is, the last two sparse matrices in the subsequence. The row sparsity of the first round is the row sparsity of the result matrix obtained by multiplying the p-1th sparse matrix in the subsequence and the pth sparse matrix in the subsequence, that is, the row sparsity of the result matrix of the intermediate subsequence of the first round.

[0062] In some embodiments, based on the above-mentioned use of a sparse adjacent list to store pointers to arrays of sparse matrices, the computing device first obtains pointers to each array of the p-th sparse matrix from the sparse adjacent list corresponding to the p-th sparse matrix, where each array is used to store the non-zero elements of the corresponding row in the p-th sparse matrix and the position information of the non-zero elements. The computing device obtains each array based on the pointer, obtains the non-zero elements of the p-th sparse matrix and the position information of each non-zero element in the sparse matrix from the array, and calculates the row sparsity of the p-th sparse matrix based on the non-zero elements and the position information of each non-zero element in the sparse matrix. Thereafter, the computing device obtains pointers to each array of the p-1-th sparse matrix from the sparse adjacent list corresponding to the p-1-th sparse matrix. The computing device obtains each array based on the pointer, obtains the non-zero elements of the p-1th sparse matrix and the position information of each non-zero element in the sparse matrix from the array, and calculates the row sparsity of the result matrix row by row based on the row sparsity of the p-1th sparse matrix and the elements in the p-1th sparse matrix. For the m-th row of the result matrix, the computing device calculates the row sparsity of the m-th row of the result matrix based on the elements in the m-th row of the p-1th sparse matrix and the row sparsity of the p-th sparse matrix, where m is an integer greater than 0. Accordingly, taking O=A×B as an example, O is the result matrix, A is the p-1th sparse matrix, and B is the p-th sparse matrix. The process of calculating the row sparsity of O row by row is as follows: Figure 5 As shown, the following steps are included: (1) the computing device obtains the row sparsity of B and A; (2) the computing device sets m to 1; (3) the computing device calculates the row sparsity of the mth row of O according to the following formula (1), and sets m to m+1; (4) if m is less than or equal to the number of rows of A, then step (3) is repeated. If m is greater than the number of rows of A, the row sparsity calculation of O is completed; (5) the computing device returns the row sparsity of O. Figure 5 The process of calculating row sparsity shown is only an example, and the computing device may also use other processes to calculate the row sparsity of the result matrix.

[0063] The above process of calculating the row sparsity of the first round is equivalent to the process of calculating the row sparsity of the multiplication result of two matrices. The embodiment of the present application uses the following formula (1) to calculate the row sparsity:

[0064]

[0065] Where O=A×B, O is used to indicate the result matrix obtained by multiplying two sparse matrices, A is used to indicate the sparse matrix on the left side of the multiplication calculation, and B is used to indicate the sparse matrix on the right side of the multiplication calculation. Used to indicate the row sparsity of the m-th row of the result matrix O, η(A,m)=0 is used to indicate the row sparsity of the m-th row of matrix A, A[m,k] is used to indicate the element in the m-th row and k-th column of matrix A, η(B,k) is used to indicate the row sparsity of the k-th row of matrix B, Π is used to indicate continuous multiplication calculation, and k is an integer greater than 0.

[0066] In the above formula (1), if the row sparsity of the m-th row of matrix A is 0, the row sparsity of the m-th row of the result matrix O is 0. If the row sparsity of the m-th row of matrix A is not 0, the computing device calculates the row sparsity of the m-th row of the result matrix O based on the elements in the m-th row of matrix A and the row sparsity of matrix B. That is, if the row sparsity of the m-th row of matrix A is not 0, the computing device obtains the column where the non-zero elements of the m-th row of matrix A are located, assigns the column number to k, and subtracts the result of multiple consecutive multiplications of (1-η(B,k)) from 1 to obtain the row sparsity of the m-th row of the result matrix O.

[0067] It should be noted that the above formula (1) is derived based on the following assumptions. Given matrices A and B, there are the following assumptions, where A is a matrix with a rows and b columns, and B is a matrix with b rows and 1 column:

[0068] (1) The positions of non-zero elements in matrices A and B are randomly and independently distributed in rows and columns;

[0069] (2) In any given row, all non-zero elements in matrix A and matrix N are uniformly and independently distributed;

[0070] (3) The probability that an element in matrix A and matrix B is non-zero is equal to the sparsity of the row in which it is located.

[0071] The row sparsity calculated by the above formula (1) has the following mathematical properties:

[0072] (1) For a given matrix multiplication O = A × B, if η(A,m)>0, then the sparsity η(O,m) of the m-th row of the output matrix O satisfies the upper and lower bounds:

[0073]

[0074] where m∈[1,a] and k∈[1,b]. The estimated row sparsity The upper and lower bounds are the same as the actual row sparsity η(O,m).

[0075] (2) Estimated row sparsity The absolute error from the true row sparsity η(O,m) satisfy:

[0076]

[0077] (3) The sparsity of the m-th row of the output matrix O can be accurately estimated when any of the following conditions are met: 1) The m-th row of the matrix A has at most one non-zero element, that is, 2) The m-th row of matrix A has at least one non-zero element A[m,k] and the k-th row of the corresponding matrix B has no zero elements, that is, η(B,k)=1.

[0078] 304. In the i-th round of calculation, the computing device calculates the row sparsity of the i-th round based on the row sparsity of the i-1-th round and the elements in the pi-th sparse matrix in the subsequence, where i is an integer greater than or equal to 2.

[0079] Among them, the row sparsity of each round is the row sparsity of the result matrix of the intermediate subsequence of that round. The intermediate subsequence of the i-th round is a sequence consisting of the pi-th sparse matrix in the sparse matrix sequence and the intermediate subsequence of the i-1-th round. The pi-th sparse matrix is also a sparse matrix in the subsequence that is adjacent to the intermediate subsequence of the i-1-th round and is located before the intermediate subsequence of the i-1-th round.

[0080] In the embodiment of the present application, the computing device uses the above formula (1) to calculate the row sparsity of the i-th round. In formula (1), O is used to indicate the result matrix of the intermediate subsequence of the i-th round, A is used to indicate the pi-th sparse matrix of the subsequence, and B is used to indicate the result matrix of the intermediate subsequence of the i-1-th round. It is used to indicate the row sparsity of the mth row of the result matrix of the intermediate subsequence of the i-th round, η(A,m) is used to indicate the row sparsity of the mth row of the pi-th sparse matrix of the subsequence, A[m,k] is used to indicate the element of the mth row and kth column of the pi-th sparse matrix of the subsequence, and η(B,k) is used to indicate the row sparsity of the kth row of the result matrix of the intermediate subsequence of the i-1-th round.

[0081] It should be noted that, in each round, the computing device does not need to calculate the result matrix of the intermediate subsequence, but only needs to calculate the row sparsity of the intermediate subsequence.

[0082] Below Figure 6 As an example, the calculation process of the row sparsity shown in steps 303 to 304 is described. Figure 6 This is an example diagram of row sparsity calculation provided by an embodiment of the present application. Figure 6 Including sparse matrices A1, A2 and A3, these three sparse matrices form a subsequence {A1, A2, A3}, with A 1,3 Indicates that A 2,3 is the intermediate subsequence composed of A2 and A3, r is used to represent the row sparsity of the matrix, The row sparsity used to represent the prediction matrix.

[0083] Among them, A1 is A2 is A3 is

[0084] The computing device obtains A through multiple rounds of computing from right to left. 1,3 The process of calculating the row sparsity of A3 is as follows: First, the computing device obtains the row sparsity of A3 (0, 1), where 0 is the row sparsity of the first row of A3 and 1 is the row sparsity of the second row. Second, the computing device uses formula (1) to calculate based on the row sparsity of A3 and the elements in A2, and obtains A 2,3 The row sparsity is (1, 1), where the first 1 is A 2,3 The first row is the row sparsity, the second 1 is A 2,3 The row sparsity of the second row. The third step is to calculate the device based on the intermediate subsequence A of the previous round. 2,3 The row sparsity of A1 and the elements in A1 are calculated using formula (1) to obtain the subsequence A 1,3 The row sparsity is (0, 1), where 0 is A 1,3 The row sparsity of the first row, 1 is A 1,3 Row sparsity for the second row.

[0085] Steps 303 to 304 above are one possible implementation method for the computing device to obtain the row sparsity of the intermediate result matrix of the subsequence based on the row sparsity of the last sparse matrix in the subsequence and the elements in the sparse matrices other than the last sparse matrix. Multiple rounds of calculations are performed according to the method shown in steps 303 to 304 above. When the intermediate subsequence of a round is the above subsequence, the row sparsity of that round is used as the row sparsity of the above subsequence.

[0086] 305. The computing device obtains the sparsity of the intermediate result matrix of the subsequence based on the row sparsity of the intermediate result matrix of the subsequence.

[0087] In an embodiment of the present application, the computing device uses the mean of the row sparsity of the intermediate result matrix as the sparsity of the intermediate result matrix, and uses the sparsity of the entire intermediate result matrix for subsequent calculations, which can reduce the amount of data in the subsequent calculation process to improve computing efficiency. In addition, obtaining the overall sparsity based on the row sparsity is equivalent to obtaining the overall sparsity according to a smaller granularity, which can improve the accuracy of the overall sparsity. In some embodiments, the computing device can also use other computing methods to calculate the sparsity of the intermediate result matrix based on the row sparsity of the intermediate result matrix, such as using the sum of the row sparsities of the intermediate result matrix as the sparsity of the intermediate result matrix, etc., and the embodiments of the present application are not limited to this.

[0088] The above steps 302 to 305 are a possible implementation method for the computing device to determine the sparsity of the intermediate result matrices of multiple subsequences in the sparse matrix sequence based on the row sparsity of multiple sparse matrices in the sparse matrix sequence. For this implementation method, in the process of obtaining multiple subsequences of the sparse matrix sequence, the computing device may first obtain all subsequences of the sparse matrix sequence, and then calculate the row sparsity of the intermediate result matrix of each subsequence. The computing device may also calculate the row sparsity of the intermediate result matrix of each subsequence every time a subsequence is obtained.

[0089] The process of calculating the row sparsity of the intermediate result matrices of multiple subsequences is performed in a serial or parallel manner. The serial manner means that the computing device calculates the row sparsity of the intermediate result matrix of one subsequence before calculating the row sparsity of the intermediate result matrix of the next subsequence. The parallel manner means that the computing device calculates the row sparsity of the intermediate result matrices of multiple subsequences simultaneously.

[0090] In the calculation process of any of the above calculation methods, the computing device first calculates a subsequence with a smaller number of sparse matrices. Since there is a relationship of inclusion and inclusion between subsequences, the calculation results of the subsequences that meet the preset conditions can be reused. The preset condition is that the subsequence is included in the subsequence currently being calculated and the subsequence is located at the rightmost side of the subsequence currently being calculated. Taking the sparse matrix sequence {A1, A2, A3, A4, A5, A6} as an example, the subsequences of the sparse matrix sequence include the subsequence {A1, A2}, the subsequence {A3, A4, A5, A6}, the subsequence {A3, A4} and the subsequence {A5, A6}, etc. The subsequence {A5, A6} is included in the subsequence {A3, A4, A5, A6} and is located at the rightmost side of the subsequence {A3, A4, A5, A6}. Therefore, the calculation result of the subsequence {A5, A6} can be reused by the subsequence {A3, A4, A5, A6}. For any subsequence with three or more sparse matrices, the computing device first searches for the calculation results of a subsequence that can be reused among the multiple subsequences for which row sparsity has been calculated. If a subsequence calculation result that can be reused exists, it is reused in the calculation process of the current subsequence. If no subsequence calculation result that can be reused exists, the calculation starts from the last sparse matrix of the current subsequence. By reusing previous calculation results, the calculation process can be reduced and the efficiency of calculating row sparsity can be improved.

[0091] The above process is illustrated by taking the example of a computing device first calculating a subsequence with a smaller number of sparse matrices. In some embodiments, the computing device first calculates a subsequence with a larger number of sparse matrices. During the calculation process, the row sparsity of the result matrix of the intermediate subsequence is used as the row sparsity of the intermediate result matrix of the same subsequence. This can avoid calculating the row sparsity of the intermediate result matrix of the same subsequence multiple times, thereby improving computing efficiency.

[0092] For the above process of calculating the sparsity of the intermediate result matrix of multiple subsequences, the embodiment of the present application provides a more efficient process, see Figure 7 , Figure 7 This is a flowchart of calculating the sparsity of an intermediate result matrix of multiple subsequences provided in an embodiment of the present application. Figure 7 A i Used to indicate the i-th sparse matrix in the sparse matrix sequence, A i，j Used to indicate that A i to A j The subsequence of these multiple sparse matrices, i and j are both integers greater than zero. Taking the sparse matrix sequence including p sparse matrices as an example, the process includes the following steps: (1) the computing device obtains the sparse matrix sequence {A1, A2, ..., A p}; (2) Set i to p; (3) If i is greater than or equal to 1, execute step (4); if i is less than 1, return the previously calculated row sparsity and sparsity; (4) Calculate A i The row sparsity and A i Sparsity; (5) Set j to p; (6) If j is greater than or equal to i+1, execute step (7); if j is less than i+1, set i to i-1 and execute step (3); (7) Calculate A i，j The row sparsity and A i，j After setting j to j-1, execute step (6).

[0093] use Figure 7 The process shown is used to calculate the sparsity of the intermediate result matrices of multiple subsequences of the sparse matrix sequence, and the row sparsity of the previously obtained intermediate result matrices can be reused as much as possible to improve the efficiency of calculating the sparsity.

[0094] 306. The computing device obtains the time consumption of the continuous multiplication of the multiple subsequences based on the sparsity of the intermediate result matrices of the multiple subsequences.

[0095] In some embodiments, the computing device first calculates the time cost of the continuous multiplication of a subsequence containing two sparse matrices, and then calculates the time cost of the continuous multiplication of a subsequence containing three sparse matrices, and so on, to calculate the time cost of the continuous multiplication of multiple subsequences. For any first subsequence, the computing device obtains the time cost of the continuous multiplication of the first subsequence based on the sparsity of the sparse matrix in the first subsequence and the sparsity of the intermediate result matrix of the first subsequence. The first subsequence is a subsequence containing two sparse matrices. For any second subsequence, the computing device divides the second subsequence according to different division methods to obtain subsequence combinations under different division methods. Each subsequence combination includes a front subsequence and a back subsequence. The front subsequence and the back subsequence in each subsequence combination do not overlap with each other. The order of the front subsequence in the second subsequence is before the back subsequence. The second subsequence is a subsequence containing at least three sparse matrices. For any subsequence combination, the computing device adds the multiplication time of the front subsequence, the multiplication time of the back subsequence, and the computing time of the multiplication of the front subsequence and the back subsequence to obtain the multiplication time of the subsequence combination, and takes the shortest multiplication time among the multiple subsequence combinations as the multiplication time of the second subsequence. The computing time of the multiplication of the front subsequence and the back subsequence is obtained based on the sparsity of the intermediate result matrix of the front subsequence, the sparsity of the intermediate result matrix of the back subsequence, and the sparsity of the intermediate result matrix of the subsequence. Accordingly, the computing device adopts the following formula (2) to obtain the multiplication time of multiple subsequences:

[0096] cost(A i,j )=min i≤c＜j {cost(A i,c )+cost(A c+1,j )+cost(A i,c ×A c+1,j )} (2)

[0097] In formula (2), A i,j Used to indicate that A i to A j This subsequence of multiple sparse matrices, cost(A i,j ) is used to indicate A i,j The multiplication time, cost(A i,c ×A c+1,j ) is used to indicate A i,c With A c+1,j The computation time of the multiplication, c is an integer greater than or equal to 2.

[0098] Taking the subsequence {A1, A2, A3} as an example, before calculating the multiplication time of the subsequence {A1, A2, A3}, the computing device has already calculated the multiplication time of the subsequences {A1, A2} and {A2, A3}. During the process of calculating the multiplication time of the subsequence {A1, A2, A3}, the computing device divides the subsequence {A1, A2, A3} into the subsequence combination {A1, A2} and A3, and the subsequence combination A1 and {A2, A3}. The multiplication time of the subsequence combination {A1, A2} and A3 is T1, and the multiplication time of the subsequence combination A1 and {A2, A3} is T2. If T1 is greater than T2, the multiplication time T2 of the subsequence combination A1 and {A2, A3} is used as the multiplication time of the subsequence.

[0099] In the above-mentioned process of obtaining the computational time of multiplying the front subsequence and the back subsequence, the computing device calls a pre-trained time model or time calculation formula, processes the sparsity of the intermediate result matrix of the front subsequence, the sparsity of the intermediate result matrix of the back subsequence, and the sparsity of the intermediate result matrix of the subsequence, and obtains the computational time of multiplying the front subsequence and the back subsequence. Since the sparsity of the intermediate result matrix can indicate the number of times the multiplication calculation is performed during the continuous multiplication process, and the number of times the multiplication calculation is performed can reflect the time of the continuous multiplication of the subsequence, the computational time of multiplying the front subsequence and the back subsequence can be predicted more accurately and quickly through the pre-trained time model or time calculation formula.

[0100] Among them, the time model is obtained based on sample data training, and can predict the computational time of multiplying the front subsequence and the back subsequence based on the sparsity of the intermediate result matrix of the front subsequence, the sparsity of the intermediate result matrix of the back subsequence, and the sparsity of the intermediate result matrix of the subsequence. The time model can be a naive Bayes classification model, a convolutional neural network model, a recurrent neural network model, and Transformers, etc., and the embodiments of the present application are not limited to this. The time calculation formula is obtained based on sample data fitting, and can be calculated based on the sparsity of the intermediate result matrix of the front subsequence, the sparsity of the intermediate result matrix of the back subsequence, and the sparsity of the intermediate result matrix of the subsequence and other parameters to obtain the computational time of multiplying the front subsequence and the back subsequence. The above-mentioned time model and time calculation formula are both a way to obtain computational time. The computing device can also obtain computational time in other ways, and the embodiments of the present application are not limited to this.

[0101] The above process of obtaining the time consumption of the continuous multiplication of the subsequences containing two sparse matrices is similar to the process of obtaining the computational time consumption of the multiplication of the front subsequence and the back subsequence, and is not described in detail in this embodiment of the present application.

[0102] The above-mentioned step 306 is a possible implementation method for the computing device to obtain the subsequence's continuous multiplication time based on the sparsity of multiple intermediate result matrices. This method uses a dynamic programming method to traverse the subsequence's continuous multiplication time under different continuous multiplication orders by dividing the subsequence, and obtain the shortest continuous multiplication time for each subsequence, which can simultaneously ensure the accuracy and efficiency of obtaining the continuous multiplication time. In other embodiments, the computing device calculates the subsequence's continuous multiplication time in a random order. For any subsequence, the computing device obtains multiple computational times for matrix multiplication calculations of the subsequence under each continuous multiplication order based on multiple continuous multiplication orders of the subsequence, sums the multiple computational times under each continuous multiplication order, and obtains the continuous multiplication time of the subsequence under multiple continuous multiplication orders. The shortest continuous multiplication time of the subsequence is determined from the multiple continuous multiplication times of the subsequence, and the shortest continuous multiplication time of the subsequence is used as the continuous multiplication time of the subsequence, thereby obtaining the continuous multiplication time of multiple subsequences. The above-mentioned process of obtaining the computational time is similar to the process of obtaining the computational time of multiplying the preceding and succeeding subsequences, and is not further described in detail in the present embodiment. In addition to the above-mentioned two methods of obtaining the computational time of multiplying subsequences, there are many other methods of obtaining the computational time of multiplying subsequences, and the present embodiment does not limit the method of obtaining the computational time of multiplying subsequences.

[0103] 307. The computing device determines the order of multiplying the multiple sparse matrices in the sparse matrix sequence based on the time consumption of multiplying the multiple subsequences through multiple rounds of partitioning process.

[0104] This step 307 includes the following steps 307A and 307B.

[0105] 307A. During the first round of division, the computing device divides the sparse matrix sequence into two subsequences, and determines the subsequence with the shorter multiplication time as the subsequence for priority execution of the multiplication operation. These two subsequences are the subsequences that make the multiplication time of the sparse matrix sequence the shortest.

[0106] In an embodiment of the present application, in the above-mentioned process of dividing the sparse matrix sequence into two subsequences, the computing device divides the sparse matrix sequence according to different division methods to obtain multiple combinations consisting of two non-overlapping subsequences, obtains the continuous multiplication time of the sparse matrix sequence under each combination, and determines the two subsequences in the combination with the shortest continuous multiplication time as the two subsequences obtained in the first round of division. Taking the sparse matrix sequence {A1, A2, A3, A4, A5, A6} as an example, according to different partitioning methods, the sparse matrix sequence can be divided into subsequences {A1, A2} and subsequences {A3, A4, A5, A6}, or the sparse matrix sequence can be divided into combinations such as subsequences {A1, A2, A3, A4} and subsequences {A5, A6}. Among them, the continuous multiplication of the combination of subsequences {A1, A2} and subsequences {A3, A4, A5, A6} takes the shortest time, so the subsequences {A1, A2} and subsequences {A3, A4, A5, A6} are determined as the two subsequences obtained in the first round of partitioning.

[0107] In the process of obtaining the multiplication time of the subsequence under each combination, the computing device can obtain the multiplication time in any of the following ways: one way is that the computing device takes the sum of the multiplication times of the two non-overlapping subsequences as the multiplication time of the subsequence; another way is that the computing device obtains the sum of the multiplication times of the two non-overlapping subsequences and the computational time of multiplying the two non-overlapping subsequences, adds the sum of the multiplication times to the computational time, and obtains the multiplication time of the subsequence. The embodiment of the present application does not limit how to obtain the multiplication time of the subsequence. The above process of obtaining the computational time is the same as the process of obtaining the computational time in step 306, and the embodiment of the present application does not limit it here.

[0108] 307B. During the nth round of division, for any subsequence divided during the n-1th round of division, the computing device divides the subsequence into a third subsequence and a fourth subsequence, and determines the subsequence with the shorter consecutive multiplication time among the third and fourth subsequences as the subsequence on which the consecutive multiplication operation is performed preferentially, the third and fourth subsequences being the subsequences that result in the shortest consecutive multiplication time for the above subsequences, and n is an integer greater than or equal to 2.

[0109] In the embodiment of the present application, the process of the computing device dividing the subsequence into the third subsequence and the fourth subsequence is similar to the process of dividing the sparse matrix sequence into two subsequences in step 307 above, and will not be further described in detail in this embodiment of the present application. After obtaining the third subsequence and the fourth subsequence, the computing device determines the subsequence of the third subsequence and the fourth subsequence that takes less time to perform the multiplication operation as the subsequence to be preferentially performed on the multiplication operation, and determines the multiplication order of the multiple sparse matrices in the sparse matrix sequence based on the priority determined during the multiple rounds of division.

[0110] The process of determining the multiplication order through multiple rounds of division processes in the above step 307 can be carried out in a variety of ways. One of the ways is that each round of division process divides all subsequences after the previous round of division. Another way is that each round of division process only divides the subsequence that was determined to be the priority for the multiplication operation in the previous round. After determining the multiplication order of the subsequence that is the priority for the multiplication operation, another subsequence is divided. For example, in the nth round of division process, the third subsequence is determined to be the subsequence that is the priority for the multiplication operation. The computing device first determines the multiplication order of the multiple sparse matrices in the third subsequence through multiple rounds of division processes, and then determines the multiplication order of the multiple sparse matrices in the fourth subsequence through multiple rounds of division processes. The embodiment of the present application does not limit the division order of the subsequences. The above-mentioned step 307 is a possible implementation method for determining the multiplication order of multiple sparse matrices in the sparse matrix sequence based on the multiplication time of multiple subsequences. This implementation method is explained by taking the example of determining the multiplication order through multiple rounds of division process. In some embodiments, the computing device obtains the multiplication time of multiple candidate multiplication sequences of the sparse matrix sequence based on the multiplication time of multiple subsequences. The candidate multiplication sequence includes a combination of multiple non-overlapping subsequences, and the candidate multiplication sequence with the shortest multiplication time among the multiple candidate multiplication sequences is determined as the multiplication order of the multiple sparse matrices in the sparse matrix sequence. In the process of obtaining the multiplication time of multiple candidate multiplication sequences, the computing device first obtains multiple candidate multiplication sequences based on the multiple subsequences of the sparse matrix sequence. For each candidate multiplication sequence, the computing device obtains the multiplication time of the candidate multiplication sequence, thereby obtaining the multiplication time of multiple candidate multiplication sequences. In the process of obtaining the multiplication time of a candidate multiplication sequence, any of the following acquisition methods can be adopted. One acquisition method is that the computing device takes the sum of the multiplication times of multiple subsequences in the candidate multiplication sequence as the multiplication time of the candidate multiplication sequence. Another acquisition method is that the computing device obtains the sum of the multiplication times of multiple subsequences in the candidate multiplication sequence and the sum of the computational time of multiplying each adjacent subsequence in the candidate multiplication sequence, and adds the sum of the multiplication times to the sum of the computational time to obtain the multiplication time of the candidate multiplication sequence. The embodiment of the present application does not limit how to obtain the multiplication time of the candidate multiplication sequence.

[0111] Steps 306 to 307 above are one possible implementation method for a computing device to determine the multiplication order of multiple sparse matrices in a sparse matrix sequence based on the sparsity of multiple intermediate result matrices. In this method, the computing device first calculates the multiplication times of all subsequences, and then determines the multiplication order based on the multiplication times of all subsequences, thereby finding the multiplication order that minimizes the multiplication time.

[0112] 308. The computing device performs a continuous multiplication operation on multiple sparse matrices in the sparse matrix sequence based on the determined continuous multiplication order.

[0113] In the embodiment of the present application, based on the determined multiplication order, the computing device can parallelize the multiplication calculations of different matrices. Taking the sparse matrix sequence {A1, A2, A3, A4, A5, A6} as an example, the multiplication order of the sparse matrix sequence is ((A1×A2)×A3)×(A4×(A5×A6)), then in the multiplication process, A1×A2 and A5×A6 can be parallelized to obtain the multiplication result A of A1×A2 and A5×A6. 1,2 and A 5,6 Afterwards, A 1,2 ×A3 and A4×A 5,6 Parallel operation: By parallelizing the multiplication of different matrices, the efficiency of multiplication of sparse matrix sequences can be improved.

[0114] In an embodiment of the present application, different sparse adjacent lists are used to store pointers to arrays of multiple sparse matrices respectively. In the process of performing a continuous multiplication operation, the computing device obtains the two sparse matrices that are first multiplied from the determined continuous multiplication order, and obtains the pointers to the arrays of the two sparse matrices from the sparse adjacent lists corresponding to the two sparse matrices. The computing device obtains each array of the two sparse matrices based on the pointer, and obtains the non-zero elements in the two sparse matrices and the position information of each non-zero element in the sparse matrix from the array. Based on the non-zero elements in the two sparse matrices and the position information of each non-zero element in the sparse matrix, the computing device calculates the intermediate result matrix obtained by multiplying the two sparse matrices according to the operation rules of matrix multiplication, stores the non-zero elements and the position information of the non-zero elements in each row of the intermediate result matrix in different arrays, stores the pointers to the different arrays in a sparse adjacent list, and continues to perform subsequent continuous multiplication operations according to the continuous multiplication order. In the subsequent multiplication process, if the above-mentioned intermediate result matrix needs to be multiplied with other sparse matrices or intermediate result matrices, the computing device obtains the pointers of each array of the intermediate result matrix from the sparse adjacent list corresponding to the intermediate result matrix, and the computing device obtains the various arrays of the intermediate result matrix based on the pointer, obtains the non-zero elements and the position information of each non-zero element in the intermediate result matrix from the array, and performs multiplication calculation based on this.

[0115] The sparse matrix multiplication method provided by the embodiment of the present application, in the process of calculating the sparsity of the intermediate result matrix, is calculated by the row sparsity of multiple sparse matrices in the sparse matrix sequence. Compared with the sparsity of the sparse matrix as a whole, the granularity of the row sparsity is smaller, which can make the accuracy of the sparsity of the intermediate result matrix higher. Sparse matrix multiplication is performed according to the sparsity with higher accuracy, which can improve the efficiency of sparse matrix multiplication. In addition, the matrix storage method provided by the embodiment of the present application can make the storage of calculation results of different rows not affect each other by storing the non-zero elements of each row of the sparse matrix in different arrays, that is, the calculation result of the current row does not need to wait until the calculation result of the previous row is fully stored before being stored, so that the calculation of different rows can be performed simultaneously, thereby improving the parallel efficiency of the multiplication calculation of different rows of the sparse matrix, and then improving the efficiency of sparse matrix multiplication.

[0116] The above describes the sparse matrix multiplication method provided by the embodiment of the present application. Figure 8 and Figure 9 , taking a computing device as an execution subject as an example, the application of the sparse matrix multiplication method provided in the embodiment of the present application in the generation of software test code is explained. In the software test code generation task, the computing device receives relevant text content for the code to be generated (test case), and the text content indicates the function of the code to be generated. Call the pre-trained large model, process the text content, and obtain the software test code. In order to improve the accuracy of the software test code generated by the large model, the information of the code library most relevant to the code to be generated can be obtained through the knowledge graph, call the pre-trained large model, process the text content and the information of the most relevant code library obtained through the knowledge graph, and obtain the software test code, which can improve the accuracy of the software test code generated by the large model.

[0117] Figure 8 This is a processing flow chart of a software test code generation task provided in accordance with an embodiment of the present application, see Figure 8 The offline part is the process before the software test code generation task is executed, and the online part is the process during the execution of the software test code generation task. The sparse matrix multiplication method provided in the embodiment of the present application is mainly used in the offline part of the relationship completion and node embedding process.

[0118] like Figure 8 As shown in the figure, after the computing device obtains the relevant text content of the code to be generated, if there is no interactive relationship between the text content and the code library, the test code generation effect of the large model will be reduced. In order to improve the test code generation effect of the large model, we will embed the process of obtaining the code library that the code to be generated may call into the processing flow of the software test code generation task. The above-mentioned acquisition of the code library that the code to be generated may call is realized through the knowledge graph. Figure 8The offline part is the process of obtaining the corresponding knowledge graph, see Figure 8 In the offline part, the computing device identifies the original data in the incremental scenario, that is, the sample data, and organizes the sample data into a database table through Schema (database mode) design. Then, through entity relationship extraction and attribute selection, the database table is converted into a graph consisting of multiple nodes and edges between nodes. The graph is a relatively rough graph for representing sample data. The computing device optimizes the graph through entity disambiguation and relationship completion so that the graph can more accurately represent the sample data, thereby obtaining the final knowledge graph. The computing device samples the final knowledge graph and uses the sampled data for node embedding, that is, the sampled data is used to train the prediction model so that the prediction model can predict the interaction relationship between nodes based on the characteristics of the nodes. The relationship completion and node embedding in the above process adopt the sparse matrix multiplication method provided in the embodiment of the present application to realize the multiplication of sparse matrices in the process of knowledge graph construction, thereby embedding the call information of the code library in the knowledge graph to obtain the final knowledge graph. In the online part, the computing device obtains the relevant text content input by the user, converts the relevant text content into computer language through the conversion module, and performs subsequent code library prediction based on the computer language. When acquiring the codebase that the code to be generated might call, the computing device connects to the prediction algorithm (calling the prediction model) to retrieve the codebase from the knowledge graph. After acquiring the codebase, the computing device can return the codebase to the user, who can select and compare the text to remove less relevant codebase. Ultimately, the codebase is assembled into a codebase that the code to be generated might call, which is then fed into the larger model, thereby improving the code generation performance of the larger model.

[0119] In the process of obtaining the knowledge graph, due to the irregularities and missing data in the original data, it is necessary to use the sparse matrix multiplication method provided in the embodiment of the present application to complete the relationship between entities, that is, to predict the interaction between nodes in the knowledge graph. The above process is usually implemented using a node embedding algorithm, that is, after obtaining the node embedding vector, the probability transfer matrix is obtained by dot multiplication to determine whether the two nodes are connected. Figure 9 For example, see Figure 9 In Figure (1), the knowledge graph includes three types of nodes representing different types of entities, namely test case nodes, code base nodes, and hardware nodes. Figure 9 The sparse matrix in (2) is used to represent the interaction relationship between nodes of different categories, that is, Figure 9The matrices M1 and M2 in the matrix are respectively used to represent the relationship between the test case and the code base, and the relationship between the code base and the hardware. If there is an interactive relationship between the two nodes, the elements in the matrices M1 and M2 are 1. The matrix M3 is a normalized matrix, which is used to make the sum of the elements in each row of the probability transfer matrix P obtained by multiplying M1 and M2 1. In the process of predicting the interactive relationship between nodes in the knowledge graph, the sparse matrix multiplication method provided in the embodiment of the present application can be used to realize the multiplication of matrices M1, M2 and M3, and the resulting matrix represents the interactive relationship between the test case and the hardware. For example, the elements in the first row and first column of the matrix P are two-thirds, indicating that test case 1 has a two-thirds probability of reaching an interactive relationship with hardware 1 along the path of "test case 1-code base 1-hardware 1".

[0120] In some embodiments, the sparse matrix multiplication method provided in the embodiments of the present application can also be applied to the calculation of paths between different nodes.

[0121] The above describes the method of the embodiment of the present application, and the following describes the device of the embodiment of the present application. It should be understood that the device described below has any function of the computing device in the above method. Figures 3 to 9 The sparse matrix multiplication method provided by the embodiment of the present application is described in detail. Based on the same inventive concept, the following will be combined with Figure 10 The following describes a sparse matrix multiplication device provided according to an embodiment of the present application. It should be understood that the technical features described in the method embodiment are also applicable to the following device embodiment.

[0122] See also Figure 10 , an embodiment of the present application provides a sparse matrix multiplication device, the sparse matrix multiplication device comprising:

[0123] A sequence acquisition module 1001 is configured to acquire a sparse matrix sequence, where the sparse matrix sequence includes multiple sparse matrices to be multiplied, and the elements in each sparse matrix are used to represent the interaction relationship between nodes;

[0124] A sparsity determination module 1002 is configured to determine the sparsity of intermediate result matrices of a plurality of subsequences in the sparse matrix sequence based on the row sparsities of the plurality of sparse matrices, where each subsequence includes at least two adjacent sparse matrices in the sparse matrix sequence, and the intermediate result matrix refers to a result of a continuous multiplication of the sparse matrices in the subsequence;

[0125] An order determination module 1003 is configured to determine a multiplication order of multiple sparse matrices in a sparse matrix sequence based on the sparsity of the multiple intermediate result matrices;

[0126] The concatenated multiplication module 1004 is configured to perform a concatenated multiplication operation on multiple sparse matrices in the sparse matrix sequence based on a concatenated multiplication order.

[0127] In some embodiments, the sparsity determination module 1002 includes:

[0128] a row sparsity obtaining unit, configured to obtain, for any subsequence, the row sparsity of an intermediate result matrix of the subsequence based on the row sparsity of the last sparse matrix in the subsequence and elements in sparse matrices other than the last sparse matrix;

[0129] The sparsity obtaining unit is used to obtain the sparsity of the intermediate result matrix of the subsequence based on the row sparsity of the intermediate result matrix of the subsequence.

[0130] In some embodiments, the subsequence includes p sparse matrices, and the row sparsity acquisition unit includes:

[0131] A first-round acquisition subunit is configured to calculate, during a first-round calculation, the row sparsity of the first round based on the row sparsity of the p-th sparse matrix in the subsequence and the elements in the p-1-th sparse matrix in the subsequence, where p is an integer greater than or equal to 2;

[0132] The i-th round acquisition subunit is used to calculate the row sparsity of the i-th round based on the row sparsity of the i-1-th round and the elements in the pi-th sparse matrix in the subsequence during the i-th round calculation process, where i is an integer greater than or equal to 2.

[0133] In some embodiments, the i-th round acquisition subunit is configured to:

[0134] If the row sparsity of the mth row of the pith sparse matrix is 0, then the row sparsity of the mth row of the i-th round is 0, where m is an integer greater than 0;

[0135] If the row sparsity of the mth row of the pith sparse matrix is not 0, then the row sparsity of the mth row of the ith round is calculated based on the elements in the mth row of the pith sparse matrix and the row sparsity of the i-1th round.

[0136] In some embodiments, the sequence determination module 1003 includes:

[0137] A continuous multiplication time acquisition unit is used to acquire the continuous multiplication time of any subsequence based on the sparsity of multiple intermediate result matrices, and obtain the continuous multiplication time of multiple subsequences;

[0138] The order determination unit is used to determine the order of multiplication of multiple sparse matrices in the sparse matrix sequence based on the time consumption of the multiplication of multiple subsequences.

[0139] In some embodiments, the sequence determination module 1003 is used to:

[0140] For any first subsequence, based on the sparsity of the sparse matrix in the first subsequence and the sparsity of the intermediate result matrix of the first subsequence, obtain the continuous multiplication time of the first subsequence, where the first subsequence is a subsequence containing two sparse matrices;

[0141] For any second subsequence, divide the second subsequence according to different division methods to obtain subsequence combinations under different division methods, each subsequence combination includes a front subsequence and a back subsequence, the front subsequence is before the back subsequence in the second subsequence, and the second subsequence is a subsequence containing at least three sparse matrices;

[0142] For any subsequence combination, based on the sparsity of the intermediate result matrix of the preceding subsequence, the sparsity of the intermediate result matrix of the following subsequence, and the sparsity of the intermediate result matrix of the second subsequence in the subsequence combination, obtain the continuous multiplication time of the subsequence combination;

[0143] The shortest consecutive multiplication time among the multiple subsequence combinations is used as the consecutive multiplication time of the second subsequence.

[0144] In some embodiments, the sequence determination unit is configured to:

[0145] In the first round of division, the sparse matrix sequence is divided into two subsequences, and the subsequence with the shorter multiplication time of the two subsequences is determined as the subsequence for which the multiplication operation is performed first. The two subsequences are the subsequences that shorten the multiplication time of the sparse matrix sequence.

[0146] In the nth round of division, for any subsequence divided in the n-1th round of division, the subsequence is divided into a third subsequence and a fourth subsequence, where the third subsequence and the fourth subsequence are the subsequences that minimize the time required for the consecutive multiplication of the subsequences, and n is an integer greater than or equal to 2;

[0147] The subsequence with the shorter consecutive multiplication time among the third subsequence and the fourth subsequence is determined as the subsequence on which the consecutive multiplication operation is preferentially performed.

[0148] In some embodiments, the sequence determination unit is configured to:

[0149] Based on the multiplication times of the multiple subsequences, obtaining the multiplication times of multiple candidate multiplication orders of the sparse matrix sequence, where the candidate multiplication orders include combinations of multiple non-overlapping subsequences;

[0150] The candidate multiplication sequence with the shortest multiplication time among the multiple candidate multiplication sequences is determined as the multiplication sequence of the multiple sparse matrices in the sparse matrix sequence.

[0151] In some embodiments, the non-zero elements of each row in the sparse matrix are stored in different arrays.

[0152] It should be understood that the sparse matrix multiplication device corresponds to the computing device in the above-mentioned method embodiment, and the modules in the device and the above-mentioned other operations and / or functions are respectively various steps and methods implemented by the computing device in order to realize the method embodiment. For specific details, please refer to the above-mentioned method embodiment. For the sake of brevity, they will not be repeated here.

[0153] In an exemplary embodiment, a computer-readable storage medium is further provided, wherein the storage medium stores at least one program code, and the program code is read by a processor to cause a computing device to perform operations such as those performed by the above-mentioned sparse matrix multiplication method. For example, the computer-readable storage medium is a non-transitory computer-readable storage medium, such as a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device.

[0154] An embodiment of the present application also provides a computer program product or computer program, which includes program code, which is stored in a computer-readable storage medium. A processor in a computing device reads the program code from the computer-readable storage medium, and the processor executes the program code, so that the computing device executes the above-mentioned sparse matrix multiplication method.

[0155] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer execution instructions, and when the device is running, the processor can execute the computer execution instructions stored in the memory to enable the chip to execute the sparse matrix multiplication method in the above-mentioned method embodiments.

[0156] Among them, the devices, equipment, computer-readable storage media, computer program products or chips provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0157] Through the description of the above embodiments, those skilled in the art can understand that, for the convenience and brevity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the sparse matrix multiplication method embodiment provided in the above embodiment belongs to the same concept. Its specific implementation process is detailed in the method embodiment and will not be repeated here.

[0158] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0159] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0160] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0161] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0162] In the description of this application, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, "at least one" means one or more, and "a plurality" means two or more. Words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not limit them to be necessarily different.

[0163] In this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0164] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, storage, and display, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the matrices involved in this application were obtained with full authorization.

[0165] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0166] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A sparse matrix multiplication method, characterized in that: The method comprises: Obtaining a sparse matrix sequence, where the sparse matrix sequence includes a plurality of sparse matrices to be multiplied, and elements in each sparse matrix are used to represent an interactive relationship between nodes; Determining, based on the row sparsities of the multiple sparse matrices, the sparsity of intermediate result matrices of a plurality of subsequences in the sparse matrix sequence, each subsequence comprising at least two adjacent sparse matrices in the sparse matrix sequence, the intermediate result matrix being a result of a continuous multiplication of the sparse matrices in the subsequence; Determining a multiplication order of multiple sparse matrices in the sparse matrix sequence based on the sparsity of the multiple intermediate result matrices; Based on the multiplication order, a multiplication operation is performed on multiple sparse matrices in the sparse matrix sequence.

2. The method according to claim 1, characterized in that The determining, based on the row sparsities of the multiple sparse matrices, of the intermediate result matrices of the multiple subsequences in the sparse matrix sequence comprises: For any subsequence, based on the row sparsity of the last sparse matrix in the subsequence and the elements in the sparse matrices other than the last sparse matrix, obtain the row sparsity of the intermediate result matrix of the subsequence; Based on the row sparsity of the intermediate result matrix of the subsequence, the sparsity of the intermediate result matrix of the subsequence is obtained.

3. The method according to claim 2, characterized in that The subsequence includes p sparse matrices, and obtaining the row sparsity of an intermediate result matrix of the subsequence based on the row sparsity of the last sparse matrix in the subsequence and elements in sparse matrices other than the last sparse matrix includes multiple rounds of calculation processes, and the multiple rounds of calculation processes include: In the first round of calculation, the row sparsity of the first round is calculated based on the row sparsity of the p-th sparse matrix in the subsequence and the elements in the p-1-th sparse matrix in the subsequence, where p is an integer greater than or equal to 2; During the i-th round of calculation, the row sparsity of the i-th round is calculated based on the row sparsity of the i-1-th round and the elements in the pi-th sparse matrix in the subsequence, where i is an integer greater than or equal to 2.

4. The method according to claim 3, characterized in that The calculating the row sparsity of the i-th round based on the row sparsity of the i-1th round and the elements in the pi-th sparse matrix in the subsequence includes: If the row sparsity of the m-th row of the pi-th sparse matrix is 0, then the row sparsity of the m-th row of the i-th round is 0, where m is an integer greater than 0; If the row sparsity of the mth row of the pith sparse matrix is not 0, the row sparsity of the mth row of the i-th round is calculated based on the elements in the mth row of the pith sparse matrix and the row sparsity of the i-1th round.

5. The method according to claim 1, wherein The determining, based on the sparsity of the plurality of intermediate result matrices, of a multiplication order of the plurality of sparse matrices in the sparse matrix sequence comprises: For any subsequence, based on the sparsity of the multiple intermediate result matrices, obtain the consecutive multiplication times of the subsequence, and obtain the consecutive multiplication times of multiple subsequences; Based on the time consumption of the continuous multiplication of the multiple subsequences, a continuous multiplication order of the multiple sparse matrices in the sparse matrix sequence is determined.

6. The method according to claim 5, characterized in that The acquiring, based on the sparsity of the plurality of intermediate result matrices, the time consumption of the continuous multiplication of the subsequences comprises: For any first subsequence, obtaining a continuous multiplication time of the first subsequence based on the sparsity of the sparse matrix in the first subsequence and the sparsity of the intermediate result matrix of the first subsequence, where the first subsequence is a subsequence including two sparse matrices; For any second subsequence, divide the second subsequence according to different division methods to obtain subsequence combinations according to the different division methods, where each subsequence combination includes a front subsequence and a back subsequence, the front subsequence precedes the back subsequence in the second subsequence, and the second subsequence is a subsequence including at least three sparse matrices; For any subsequence combination, based on the sparsity of the intermediate result matrix of the first subsequence, the sparsity of the intermediate result matrix of the second subsequence, and the sparsity of the intermediate result matrix of the second subsequence in the subsequence combination, obtaining the continuous multiplication time of the subsequence combination; The shortest consecutive multiplication time among the multiple subsequence combinations is used as the consecutive multiplication time of the second subsequence.

7. The method according to claim 5, characterized in that Determining the order of the continuous multiplication of the plurality of sparse matrices in the sparse matrix sequence based on the time consumption of the continuous multiplication of the plurality of subsequences includes a multi-round division process, wherein the multi-round division process includes: In a first round of division, the sparse matrix sequence is divided into two subsequences, and the subsequence with the shorter consecutive multiplication time of the two subsequences is determined as the subsequence on which the consecutive multiplication operation is preferentially performed, wherein the two subsequences are the subsequences that minimize the consecutive multiplication time of the sparse matrix sequence; In the nth round of division, for any subsequence divided in the (n-1)th round of division, the subsequence is divided into a third subsequence and a fourth subsequence, where the third subsequence and the fourth subsequence are subsequences that minimize the time required for consecutive multiplication of the subsequence, where n is an integer greater than or equal to 2; The subsequence with the shorter consecutive multiplication time among the third subsequence and the fourth subsequence is determined as the subsequence on which the consecutive multiplication operation is preferentially performed.

8. The method according to claim 5, characterized in that The determining, based on the time consumption of the continuous multiplication of the multiple subsequences, a continuous multiplication order of the multiple sparse matrices in the sparse matrix sequence includes: Based on the consecutive multiplication times of the multiple subsequences, obtaining consecutive multiplication times of multiple candidate consecutive multiplication orders of the sparse matrix sequence, where the candidate consecutive multiplication orders include a combination of multiple non-overlapping subsequences; The candidate multiplication order with the shortest multiplication time among the multiple candidate multiplication orders is determined as the multiplication order of the multiple sparse matrices in the sparse matrix sequence.

9. The method according to any one of claims 1 to 8, characterized in that The non-zero elements of each row in the sparse matrix are stored in different arrays.

10. A sparse matrix multiplication device, characterized in that: The device comprises: A sequence acquisition module is used to acquire a sparse matrix sequence, wherein the sparse matrix sequence includes multiple sparse matrices to be multiplied, and the elements in each sparse matrix are used to represent the interactive relationship between nodes; a sparsity determination module, configured to determine the sparsity of intermediate result matrices of a plurality of subsequences in the sparse matrix sequence based on the row sparsities of the plurality of sparse matrices, each subsequence comprising at least two adjacent sparse matrices in the sparse matrix sequence, the intermediate result matrix being a result of a continuous multiplication of the sparse matrices in the subsequence; An order determination module, configured to determine a multiplication order of the plurality of sparse matrices in the sparse matrix sequence based on the sparsity of the plurality of intermediate result matrices; A concatenated multiplication module is configured to perform a concatenated multiplication operation on a plurality of sparse matrices in the sparse matrix sequence based on the concatenated multiplication order.

11. A computing device, characterized in that The computing device comprises a processor configured to execute program code, so that the computing device performs the method according to any one of claims 1 to 9.

12. A computing device cluster, characterized in that: The method comprises a plurality of computing devices including a processor, wherein the processor is configured to execute program code, so that the computing device cluster performs the method according to any one of claims 1 to 9.

13. A computer-readable storage medium, characterized in that At least one program code is stored in the storage medium, and the at least one program code is read by a processor to enable a computing device to execute the method according to any one of claims 1 to 9.

Images