Sparse matrix feature extraction method and device

Abstract

The invention discloses a sparse matrix feature extraction method and device, and relates to the technical field of machine learning. The computing device extracts expert features of the sparse matrix, the expert features are generally low-calculation-amount features, and the expert features reflect local features of the sparse matrix; the computing device also extracts a plurality of matrix digests of the sparse matrix using a plurality of resolutions, since elements included in the matrix digests of the sparse matrix are determined according to a plurality of sub-matrices determined by the resolutions, which means that the matrix digests can describe global features of the sparse matrix. Thus, the computing device optimizes the multiple expert features through the matrix abstract describing the global features of the sparse matrix, the problem that the expert features are uneven is solved, and the feature redundancy of the general features of the sparse matrix is reduced.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a method and device for extracting features from a sparse matrix. Background Art

[0002] Sparse matrices arise in almost all large-scale scientific and engineering computing fields, including computational fluid dynamics, statistical physics, circuit simulation, image processing, and nanomaterial computing. For big data application analysis and deep learning, large-scale sparse linear equations and their solutions are becoming increasingly common. In large-scale sparse matrix scenarios, since the sparse matrix contains a large number of zero elements, it is both a waste of computing resources and meaningless for the solver to directly access the entire matrix, so dimensionality reduction or feature extraction of the sparse matrix is required. In related technologies, simple feature extraction is performed on the sparse matrix based on expert experience, and the model is trained based on the extracted features. However, the expert features determined based on expert experience are of varying quality, and there may be significant feature redundancy between expert features. Summary of the Invention

[0003] The present application provides a feature extraction method and device for a sparse matrix, which solves the problem of large feature redundancy between expert features caused by the uneven quality of expert features determined based on expert experience, and is conducive to improving the efficiency of model training using the common features of sparse matrices.

[0004] This application adopts the following technical solution.

[0005] In a first aspect, the present application provides a method for extracting features from a sparse matrix. The method can be executed by a computing device or a chip or processor included in the computing device. The method is described by taking a computing device executing the feature extraction method provided by the present application as an example. The feature extraction method includes: the computing device obtains a training set including a sparse matrix, extracts multiple expert features of the sparse matrix, and extracts multiple matrix summaries of the sparse matrix using multiple resolutions; one resolution corresponds to one matrix summary, and the matrix summary includes multiple elements, each element corresponding to a submatrix in the sparse matrix. Furthermore, the computing device processes multiple expert features based on the multiple matrix summaries to obtain a common feature of the sparse matrix, wherein the feature redundancy of the common feature of the sparse matrix is less than or equal to the feature redundancy of the multiple expert features.

[0006] In this application, a computing device not only extracts expert features of a sparse matrix, which are typically low-computation features and reflect the local characteristics of the sparse matrix, but also uses multiple resolutions to extract multiple matrix summaries of the sparse matrix. Because the matrix summary of the sparse matrix includes elements determined by multiple submatrices determined by the resolution, this means that the matrix summary can describe the global characteristics of the sparse matrix. In this way, the computing device optimizes the matrix summaries that describe the global characteristics of the sparse matrix to multiple expert features, which is beneficial for improving the problem of uneven quality of expert features and reducing feature redundancy of the common characteristics of the sparse matrix.

[0007] Moreover, when the computing device uses the general features of the sparse matrix for model training, since the feature redundancy of the general features of the sparse matrix is less than or equal to the feature redundancy of multiple expert features, the amount of data of the features required to be processed for model training is reduced, which is conducive to improving the efficiency of model training.

[0008] In an optional implementation, the computing device processes multiple expert features based on multiple matrix summaries to obtain universal features, including: the computing device filters the multiple expert features using at least two methods to obtain at least two feature sets, wherein one of the methods corresponds to one feature set, and the at least two methods include two or three of dependency ensemble extraction, supervised ensemble extraction, and causal analysis. Furthermore, the computing device corrects the at least two feature sets to obtain a candidate universal feature set, the candidate universal feature set including multiple candidate universal features, and processes the candidate universal feature set based on the multiple matrix summaries to obtain universal features of the sparse matrix.

[0009] In a possible example, the dependency integration extraction indicates that: multiple expert features are ranked by their feature importance to the model, and a feature set is selected according to a ranking gradient, where the feature importance indicates the accuracy improvement effect of the feature on the model.

[0010] In another possible example, the supervised ensemble extraction indicates: sorting the improvement degree of the model accuracy of the model by the multiple expert features, and selecting the top N ones, where N is a positive integer.

[0011] In another possible example, the causal analysis indicates that a feature set that has a positive improvement on the model accuracy of the model is selected from the plurality of expert features.

[0012] In the present application, in the process of obtaining the universal features of the sparse matrix, the computing device not only uses multiple matrix summaries that characterize the global features of the sparse matrix, but also uses multiple different methods to screen the expert features to obtain a candidate universal feature set. Since the expert features characterize the local features of the sparse matrix, the computing device processes the candidate universal feature set with multiple matrix summaries, which is actually to optimize the screened local features using the global features of the sparse matrix. Therefore, the universal features obtained by the computing device can comprehensively characterize the sparse matrix and reduce the feature redundancy of the universal features.

[0013] In an optional implementation, the sparse matrix includes multiple sparse matrices, and the candidate universal feature set includes: candidate universal feature sets corresponding to different sparse matrices. The aforementioned computing device processes the candidate universal feature set according to the multiple matrix summaries to obtain universal features of the sparse matrix, including: the computing device processes the candidate universal feature set of the first sparse matrix according to the multiple matrix summaries of the first sparse matrix to obtain universal features of the first sparse matrix, where the first sparse matrix is any one of the multiple sparse matrices. And, the computing device corrects multiple groups of universal features of the multiple sparse matrices to obtain universal features of the sparse matrix.

[0014] In this application, the computing device will be calibrated according to the common features of different sparse matrices, so that the final common features can be applied to the feature extraction process of sparse matrices including different elements, avoiding the problem of the feature extraction process being unable to adapt due to differences in the elements included in the sparse matrix, and improving the generalization of these common features.

[0015] In an optional implementation, the process of extracting the plurality of expert features of the sparse matrix by the computing device includes: the computing device inputting the sparse matrix into an expert system to obtain the plurality of expert features of the sparse matrix. The expert system includes a feature extraction layer for extracting the expert features of the matrix.

[0016] In a second aspect, the present application provides a feature extraction device for a sparse matrix. The feature extraction device can be applied to a computing device or a chip or processor included in the computing device, such as the feature extraction device including a module or software unit for executing the method of the first aspect or any optional implementation of the first aspect.

[0017] Exemplarily, the feature extraction device provided by the present application includes: an acquisition module, an extraction module, and a feature processing module. The acquisition module is used to acquire a training set, which includes a sparse matrix. The extraction module is used to extract multiple expert features of the sparse matrix and extract multiple matrix summaries of the sparse matrix using multiple resolutions. One resolution corresponds to a matrix summary, and a matrix summary includes multiple elements, each element corresponding to a submatrix in the sparse matrix. The feature processing module is used to process multiple expert features based on the multiple matrix summaries to obtain common features of the sparse matrix, and the feature redundancy of the common features of the sparse matrix is less than or equal to the feature redundancy of the multiple expert features.

[0018] In conjunction with the feature extraction device provided in the second aspect, in an optional implementation, the feature processing module is specifically configured to: screen multiple expert features using at least two methods to obtain at least two feature sets. One method corresponds to a set of feature sets, and the at least two methods include two or three of dependency ensemble extraction, supervised ensemble extraction, and causal analysis. The feature processing module is specifically configured to: correct the at least two sets of feature sets to obtain a candidate universal feature set, wherein the candidate universal feature set includes multiple candidate universal features. And, process the candidate universal feature set according to multiple matrix summaries to obtain universal features of the sparse matrix.

[0019] In combination with the feature extraction device provided in the second aspect, in an optional implementation method, the dependency integration extraction indicates: sorting the feature importance of multiple expert features to the model, and selecting a feature set according to the ranking gradient, and the feature importance indicates the effect of the feature on improving the accuracy of the model. The supervised integration extraction indicates: sorting the improvement degree of the model accuracy of the model by multiple expert features, and selecting the top N ranked ones, where N is a positive integer. The causal analysis indicates: selecting a feature set that has a positive improvement on the model accuracy of the model from multiple expert features.

[0020] In conjunction with the feature extraction device provided in the second aspect, in an optional implementation, the sparse matrix includes multiple sparse matrices, and the candidate universal feature set includes: candidate universal feature sets corresponding to different sparse matrices. The feature processing module is specifically configured to: process the candidate universal feature set of the first sparse matrix based on multiple matrix summaries of the first sparse matrix to obtain universal features of the first sparse matrix, where the first sparse matrix is any one of the multiple sparse matrices; and correct multiple groups of universal features of the multiple sparse matrices to obtain universal features of the sparse matrices.

[0021] In conjunction with the feature extraction device provided in the second aspect, in an optional implementation, the extraction module is specifically configured to: input a sparse matrix into an expert system to obtain a plurality of expert features of the sparse matrix. The expert system includes a feature extraction layer configured to extract the expert features of the matrix.

[0022] In combination with the feature extraction method provided in the first aspect or the feature extraction device provided in the second aspect, in an optional implementation, an element included in the aforementioned matrix summary is: an element in the submatrix, or a value obtained after processing the element in the submatrix.

[0023] In combination with the feature extraction method provided in the first aspect or the feature extraction device provided in the second aspect, in an optional implementation, the common features of the aforementioned sparse matrix include two or more of the following features: the proportion of non-zero elements, the measure of symmetry, the diagonal minimum, the average value of non-zero elements, numerical symmetry, the upper triangular bandwidth, the percentage of data symmetry, the average continuous non-zero block, the Frobenius norm of the symmetric part, the Frobenius norm of the asymmetric part, the maximum ratio of each row, the maximum ratio of each column, the standard deviation of the sparsest row, the minimum value of the densest row, the minimum positive value or maximum negative value of the diagonal elements minus the sum of the absolute non-diagonal elements, and the sum of the non-zero elements.

[0024] In a third aspect, the present application provides a chip. The chip includes a control circuit and an interface circuit. The interface circuit is configured to obtain a training set and, in conjunction with the control circuit, implement the steps of the method of the first aspect or any optional implementation of the first aspect.

[0025] In a fourth aspect, the present application provides a computing device. The computing device includes a processor and a memory. The memory is configured to store a set of computer instructions. When the processor executes the set of computer instructions, the steps of the method according to the first aspect or any optional implementation of the first aspect are implemented.

[0026] In a fifth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium includes computer instructions. When the computer instructions are executed in a computing device, the computing device implements the steps of the method of the first aspect or any optional implementation of the first aspect.

[0027] In a sixth aspect, the present application provides a computer program product. When the computer program product is executed in a computing device, the computing device implements the steps of the method of the first aspect or any optional implementation of the first aspect.

[0028] Regarding the beneficial effects of the technical solutions provided in aspects 2 to 6, reference may be made to the description of aspect 1 or any optional implementation of aspect 1, and no further description is given here. Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of the architecture of a model training system provided in this application;

[0030] Figure 2 A schematic diagram of the model training process provided in this application;

[0031] Figure 3 Schematic diagram of the process of feature extraction method of sparse matrix provided in this application Figure 1 ;

[0032] Figure 4 A schematic diagram of obtaining the matrix summary provided for this application;

[0033] Figure 5 Schematic diagram of the process of feature extraction method of sparse matrix provided in this application Figure 2 ;

[0034] Figure 6 A schematic diagram of the visualization results of a causal analysis provided in this application;

[0035] Figure 7 Schematic diagram of the generalization of the common features provided in this application on different datasets;

[0036] Figure 8 A schematic diagram of the structure of a sparse matrix feature extraction device provided in this application;

[0037] Figure 9 A schematic diagram of the structure of a computing device provided in this application. DETAILED DESCRIPTION

[0038] The present application provides a method for extracting features from a sparse matrix. A computing device not only extracts expert features from the sparse matrix, which are typically low-computation features and reflect the local features of the sparse matrix, but also extracts multiple matrix summaries of the sparse matrix using multiple resolutions. Since the matrix summary of the sparse matrix includes elements determined by multiple submatrices determined by the resolution, this means that the matrix summary can describe the global features of the sparse matrix. In this way, the computing device optimizes the matrix summaries that describe the global features of the sparse matrix against multiple expert features, which is beneficial for improving the uneven quality of expert features and reducing feature redundancy in the common features of the sparse matrix.

[0039] Moreover, when the computing device uses the general features of the sparse matrix for model training, since the feature redundancy of the general features of the sparse matrix is less than or equal to the feature redundancy of multiple expert features, the amount of data of the features required to be processed for model training is reduced, which is conducive to improving the efficiency of model training.

[0040] The technical solutions provided in this application can be applied not only to current scenarios such as model training or feature extraction, but also to future model training or feature extraction technologies. The terms used in the implementation methods of this application are only used to explain the specific embodiments of this application and are not intended to limit this application. The following is a brief introduction to some concepts that may be involved in this application.

[0041] Sparse matrix: If the number of zero elements is much greater than the number of non-zero elements, and the non-zero elements are distributed irregularly, the matrix is called a sparse matrix. On the contrary, if the number of non-zero elements is the majority, the matrix is called a dense matrix. The density of a matrix is defined as the total number of non-zero elements divided by the total number of elements in the matrix.

[0042] Matrix summary: This refers to the vector extracted when a computing device performs feature extraction on a matrix at a certain resolution. The size of the matrix summary is consistent with the resolution used to obtain it. For example, if the resolution is 10×10, the matrix summary corresponding to that resolution is a 10×10 matrix.

[0043] Solver: A solver that uses mathematical models and algorithms to find the optimal solution or an approximation of the optimal solution.

[0044] Pre-conditioner: It makes the iterative algorithm converge faster by modifying the ill-conditioned system.

[0045] Model evaluation: Artificial intelligence (AI) training ultimately produces an algorithm model. Model evaluation is the process of evaluating the performance of this algorithm model on a test dataset. Commonly used metrics include precision, accuracy, and recall.

[0046] Principal component analysis (PCA) is one of the most widely used data dimensionality reduction algorithms. The main idea of PCA is to map n-dimensional features to k-dimensional features. These k-dimensional features are new orthogonal features, also known as principal components. They are reconstructed from the original n-dimensional features.

[0047] Feature importance: It is used to understand the usefulness or value of each feature (variable or input) for making predictions. The goal is to identify the most important features that have the greatest impact on the model output. It is a method often used in machine learning.

[0048] Feature selection: Filter model features based on correlation and return a set of features with the highest correlation in the dataset, with a variable number of features.

[0049] Causal inference: Causal inference is the process of determining the actual, independent effects (effects) of a given phenomenon (cause) within a larger system.

[0050] The implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0051] Figure 1 This is a schematic diagram of the architecture of a model training system provided in this application. Figure 1 As shown, the model training system 100 includes a computing device 110 , a computing device 120 , a database 130 , a terminal device 140 , a data storage system 150 and a data acquisition device 160 .

[0052] The computing device 110 may be a terminal, such as a computer, a mobile phone terminal, a tablet computer, a laptop computer, a virtual reality (VR) device, an augmented reality (AR) device, a mixed reality (MR) device, an extended reality (ER) device, a camera, or an in-vehicle computer, or an edge device (e.g., a box with a processing chip). In the present application, the computing device 110 may be a computing device connected to a base station, or a computing device deployed at a base station, such as a server or a cloud device.

[0053] The computing device 120 may be a terminal, or other computing device that supports integer computing, such as a server or a cloud device.

[0054] As a possible embodiment, computing device 110 and computing device 120 are deployed on different physical devices (such as servers or servers in a cluster), or computing device 110 and computing device 120 are different physical devices. Exemplarily, computing device 110 and computing device 120 are processors deployed on different physical devices. For example, computing device 110 can be a graphics processing unit (GPU), a central processing unit, other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor, etc. Computing device 120 can be a graphics processing unit (GPU), a neural network processing unit (NPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0055] In another possible embodiment, the computing device 110 and the computing device 120 are deployed on the same physical device, or the computing device 110 and the computing device 120 are the same physical device.

[0056] Data acquisition device 160 is used to collect training data and store the training data in database 120. Data acquisition device 160 and computing device 110 and computing device 120 may be the same or different devices. In the embodiment of the present application, data acquisition device 160 may be a video camera, a camera, a mobile phone, a tablet computer, or a computing device or network device with image acquisition capabilities.

[0057] Computing device 110 is used to train the neural network using training data until the loss function in the neural network converges. When the loss function value is less than a specific threshold, neural network training is completed, thereby achieving a certain accuracy. Computing device 110 then transfers the trained neural network 101 to computing device 120. Computing device 110 is used to implement model training for document images containing text content based on the trained neural network 101.

[0058] In this embodiment, the neural network model may be referred to as a model training model. When the model training model is used to recognize handwritten Chinese characters, the model training model may also be referred to as a Chinese character recognition model.

[0059] Optionally, neural network 101 is used to recognize text content in a document image and output a model training result for the document image. Neural network 101 can be a network type suitable for model training. For example, neural network 101 can be a convolutional neural network (CNN), a recurrent neural network, or a graph neural network.

[0060] In some embodiments, computing device 110 and computing device 120 are the same computing device. The computing device can configure the trained neural network 101 to itself and use the trained neural network 101 to implement the target functions to be achieved by the model, such as functions such as identification or document verification in commercial areas, schools, parks, and stadiums in cities, and operations such as target detection, object recognition or classification on data, as well as functions such as face-swiping payment and object classification (such as commodity classification).

[0061] In other embodiments, computing device 110 may deploy the trained neural network 101 to multiple computing devices, so that each computing device can implement the target functionality of the aforementioned model. In addition to the target functionality of the aforementioned embodiments, neural network 101 may also implement some of the functionality achieved by a large language model (LLM), among other capabilities.

[0062] It should be noted that, in actual applications, the training data maintained in database 130 does not necessarily come from data acquisition device 160, but may also be received from other devices. In addition, computing device 110 does not necessarily train the neural network entirely based on the training data maintained in database 130, but may also obtain training data from the cloud or other places to train the neural network. The above description should not be construed as limiting the embodiments of the present application.

[0063] Furthermore, according to the functions performed by the computing device 110, the computing device 110 can be further divided into Figure 1 The architecture shown, such as Figure 1 As shown, the computing device 110 is configured with a computing module 111 , an I / O interface 112 and a pre-processing module 113 .

[0064] Taking the computing device 110 as an example of a computing device connected to an access site such as a base station or a wireless access point, the computing module 111 can be a GPU, CPU, other general-purpose processors, DSP, ASIC, FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. on an on-board computer.

[0065] For example, the computing module 111 is used to run the neural network 101 to implement a feature extraction function of a sparse matrix to obtain expert features or a matrix summary of the sparse matrix.

[0066] I / O interface 112 is used to exchange data with external devices. A user can input data into I / O interface 112 through terminal device 140, such as instructions for instructing computing device 110 to initiate feature extraction or model training on training data. Alternatively, the input data can come from database 130.

[0067] The preprocessing module 113 is configured to preprocess the input data received by the I / O interface 112. In an embodiment of the present application, the preprocessing module 113 can be configured to generate training data, such as a training set, a validation set, and a test set, based on the input data received from the I / O interface 112. Optionally, the preprocessing module 113 can also perform preprocessing operations such as denoising on the input data, such as matrix data and graph data of a base station, to eliminate irrelevant information and restore useful real information.

[0068] When the computing device 110 preprocesses input data, or when the computing module 111 of the computing device 110 performs calculations and other related processing, the computing device 110 can call data, codes, etc. in the data storage system 150 for corresponding processing, and can also store data and instructions obtained from the corresponding processing in the data storage system 150.

[0069] Finally, the I / O interface 112 returns the processing result to the terminal device 140, thereby providing it to the user so that the user can view the processing result. In the embodiment of the present application, the processing result can be a common feature of a sparse matrix or a trained model.

[0070] The terminal device 140 can serve as a data acquisition terminal, collecting input data input into the I / O interface 112 and processing results output from the I / O interface 112 as new sample data, and storing the data in the database 130. Of course, the sample data can also be collected without passing through the terminal device 140, and instead the I / O interface 112 can store the input data input into the I / O interface 112 and processing results output from the I / O interface 112 as new sample data in the database 130.

[0071] Figure 1This is only a schematic diagram of a system architecture provided by an embodiment of the present application. Figure 1 The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. According to the user's requirements for model training, the model training system and computing equipment may include more or fewer hardware components, and this application does not limit this. Figure 1 In the embodiment, the data storage system 150 is an external memory relative to the computing device 110. In other cases, the data storage system 150 may also be placed in the computing device 110.

[0072] The scenarios in which this application can be applied include, but are not limited to: scenarios such as recommending the optimal storage method for sparse matrices, scenarios for large-scale model tuning, and scenarios involving model training or model usage for feature extraction of sparse matrices, etc.

[0073] Below Figure 1 Based on the model training system and computing device shown, the training process of the intelligent solver model is explained by taking the model of intelligent solver combination recommendation (multi-classification task) as an example. Figure 2 This application provides a flowchart of a model training process. The model training process includes several parts: matrix feature extraction, AI model and optimal solver combination. The contents of each part are explained below.

[0074] 1. Matrix feature extraction.

[0075] The computing device extracts the expert features and the original features of the sparse matrix respectively. In some optional implementations, the expert features and the original features of the sparse matrix can also be referred to as the complete features of the sparse matrix.

[0076] The complete feature extraction of sparse matrices consists of two parts: the first part is to finely screen the low-computational expert features extracted from existing expert experience to form a set of generalizable features; the second part is to summarize the original information of the matrix through multi-resolution extraction to further supplement the expert features.

[0077] Among them, the computing device, based on the low-computation expert features extracted based on the existing expert experience, systematically screens these low-computation expert features to obtain the expert features of the sparse matrix. For example, the methods adopted for systematic feature screening may include but are not limited to: dependency integration extraction, supervisory integration extraction and causal analysis. For the specific method of extracting expert features, please refer to the following Figures 3 to 6 The embodiments of the present invention are not described in detail here.

[0078] The original features can be: the matrix summary obtained by the computing device using multiple resolutions to extract the underlying information of the sparse matrix. The specific extraction method of the matrix summary representing the original features can be referred to below. Figure 4 The embodiments of the present invention are not described in detail here.

[0079] 2. AI model.

[0080] Exemplarily, the computing device inputs one or both of the expert features and original features obtained in the matrix feature extraction into the AI model, and trains the AI model to improve the accuracy of the AI model.

[0081] 3. Optimal solver combination.

[0082] When solving sparse matrix linear equations, the performance of different solver and preconditioner combinations varies significantly, making exhaustive attempts prohibitively expensive. To improve computational efficiency, computing devices utilize AI models to tailor solver and preconditioner recommendations based on matrix characteristics, recommending the optimal solver combination based on matrix characteristics.

[0083] Regarding the above matrix feature extraction process, the following Figure 1 Based on the computing device shown, a sparse matrix feature extraction method is provided, such as Figure 3 As shown, Figure 3 Schematic diagram of the process of feature extraction method of sparse matrix provided in this application Figure 1 The feature extraction method can be executed by a computing device or a chip in the computing device. The embodiment of the present application takes the feature extraction method executed by a computing device as an example. For example, the computing device can be Figure 1 The computing device 110 or computing device 120 shown in FIG. Figure 3 The feature extraction method provided in the embodiment of the present application includes the following S310 to S330.

[0084] S310: The computing device obtains a training set.

[0085] The training set includes sparse matrices used for training.

[0086] In one possible scenario, the training set includes a sparse matrix, such as a first sparse matrix.

[0087] In another possible scenario, the training set includes multiple sparse matrices, such as a first sparse matrix, a second sparse matrix, and other possible sparse matrices.

[0088] In this embodiment, the present application does not limit the number of sparse matrices included in the training set.

[0089] As an optional example, the aforementioned training set is stored in a memory in the computing device.

[0090] As another optional example, the aforementioned training set is read by the computing device from a storage device that communicates with the computing device.

[0091] As another optional example, the aforementioned training set is sent to the computing device by other devices.

[0092] The above three examples are merely optional methods of S310 provided in this embodiment and should not be construed as limiting this application. In some feasible examples, the computing device may also generate the above training set while running the model or application; or the user may input the sparse matrix included in the above training set into the computing device through an input device.

[0093] S320: The computing device extracts multiple expert features of the sparse matrix, and extracts multiple matrix summaries of the sparse matrix using multiple resolutions.

[0094] Optionally, the expert features are matrix features determined based on historical training data, such as low-computation features extracted based on expert experience.

[0095] Exemplarily, the computing device inputs the sparse matrix into the expert system, and the expert system processes the sparse matrix to obtain a plurality of expert features. For example, the expert system includes a feature extraction layer, which is used to extract expert features of the matrix.

[0096] The feature extraction layer can be implemented by a convolutional neural network (CNN). The feature extraction layer may include at least one convolutional layer, which includes at least one convolution unit. The convolutional layer can be used to extract various expert features of the sparse matrix. The expert features are the data output by the convolution layer, activation layer, pooling layer or batch normalization layer in the feature extraction layer. In some possible examples, the feature extraction layer may also include an activation layer, for example, a rectified linear units layer (ReLU), or a parametric rectified linear unit (PReLU). In other possible examples, the convolutional neural network may also include other functional modules such as a pooling layer, a batch normalization layer (BN layer), and a fully connected layer. For the relevant principles of the various functional modules of CNN, please refer to the relevant explanations of the prior art and will not be repeated here.

[0097] This article provides an example of a convolutional layer within a feature extraction layer. A convolutional layer refers to the layer of neurons in a convolutional neural network that performs convolution processing on the input signal. A convolutional layer can include multiple convolution operators, also known as kernels. In image processing, a convolution operator acts as a filter that extracts specific information from the input image matrix. Essentially, a convolution operator is a weight matrix, which is typically predefined. During the convolution operation, the weight matrix is typically applied horizontally, pixel by pixel (or two pixels by two pixels, depending on the stride), to the input image, extracting specific features from the image. Different weight matrices can be used to extract different features from the image. When a convolutional neural network has multiple convolutional layers, the initial convolutional layers often extract more general features, also known as low-level features. As the depth of the convolutional neural network increases, the features extracted by subsequent convolutional layers become increasingly complex, such as high-level semantic features. Features with higher semantics are more applicable to the problem being solved. For information on other layers that may be included in the feature extraction layer, please refer to the general technical description and will not be elaborated here.

[0098] In this way, the above-mentioned expert features extracted by the computing device are all low-computation features, which avoids the problem of feature extraction occupying more processing resources in the computing device, and is conducive to improving the feature extraction efficiency of sparse matrices.

[0099] In this embodiment, one resolution corresponds to one matrix summary, and one matrix summary includes multiple elements. A sparse matrix has multiple sub-matrices determined by one resolution, and one element corresponds to one sub-matrix.

[0100] The resolution is used to indicate the row and column information used by the computing device to extract matrix information, such as the resolution is 10×10, 100×100, 10×20, 1×100 or others.

[0101] A matrix summary is a vector extracted from a matrix by a computing device during feature extraction. This matrix summary represents the global characteristics of the matrix. The size of the matrix summary is consistent with the resolution used to obtain it. For example, if the resolution is 10×10, the corresponding matrix summary is a 10×10 matrix. If the resolution is 1×100, the corresponding matrix summary is a 1×100 matrix (vector).

[0102] In an optional example, an element included in the matrix summary is an element in the submatrix. For example, the computing device selects any element from the submatrix as the value of the element corresponding to the submatrix in the matrix summary, for example, the any element is the center element of the submatrix, an element on an edge, or another element between an edge and the center of the submatrix.

[0103] like Figure 4 As shown in ①, the resolution is 4×4, and the principle of extracting the matrix summary is to extract the non-zero maximum value in the submatrix determined by the resolution, and the obtained matrix summary is matrix summary 1.

[0104] Another example Figure 4 As shown in ③, the resolution is 2×2, and the principle of extracting the matrix summary is to extract the non-zero maximum value in the submatrix determined by the resolution, and the obtained matrix summary is matrix summary 3.

[0105] In another optional example, one element of the matrix summary is a value obtained by processing the elements in the submatrix corresponding to the element. The processing here may refer to taking the average, median, or maximum value of the elements in the submatrix, etc., which is not limited in this application.

[0106] like Figure 4 As shown in ②, the resolution is 2×1, and the principle of extracting the matrix summary is to extract the average value in the submatrix determined by the resolution. The resulting matrix summary is matrix summary 2.

[0107] The above-mentioned methods for obtaining the elements included in the matrix summaries are merely examples provided in the embodiments of the present application and should not be construed as limiting the present application. As a feasible specific implementation method, the method in which the computing device extracts the sparse matrix using different resolutions includes: the computing device uses a multi-scale pooling method to extract the underlying information of the sparse matrix. Multi-scale pooling is to extract the underlying information of the matrix using different resolutions. Under the superposition of multiple layers of pooling, the information representing the matrix (matrix summary) can be more complete.

[0108] It is worth noting that in the above S320, the process of the computing device extracting expert features and the process of obtaining the matrix summary can occur simultaneously or successively. This application does not limit whether the computing device obtains the expert features first or the matrix summary first.

[0109] S330: The computing device processes the multiple expert features according to the multiple matrix summaries to obtain common features of the sparse matrix.

[0110] In this embodiment, the feature redundancy of the common features of the sparse matrix is less than or equal to the feature redundancy of the multiple expert features.

[0111] Illustratively, Table 1 below provides a possible example, and the common features of a sparse matrix include two or more features in Table 1 below.

[0112] Table 1

[0113]

[0114] The number of non-zero elements (nnz_frac) is the ratio of the number of non-zero elements in the sparse matrix to all elements in the sparse matrix.

[0115] Symmetry measure (sym_det): indicates the symmetry of the arrangement of elements in a sparse matrix.

[0116] Diagonal minimum (diag_min): refers to the minimum value of the non-zero elements on the main diagonal of the sparse matrix.

[0117] The average value of non-zero elements (avg_nnz_norm): refers to the average of all non-zero elements in the sparse matrix.

[0118] Numerical symmetry (sym_s): refers to the similarity of each element in a sparse matrix along the diagonal, horizontal axis, or vertical axis.

[0119] upper triangular bandwidth (upper_bw) is a nonnegative integer scalar.

[0120] Numeric (data) symmetry percentage (symm_num_percent): indicates the proportion of pairs of elements in the sparse matrix that are identical along the diagonal, horizontal axis, or vertical axis to all pairs of elements in the sparse matrix.

[0121] The average number of consecutive non-zero blocks (avg_blocks) is used to indicate the distribution of non-zero elements in a sparse matrix.

[0122] The Frobenius norm (F-norm) is the square root of the sum of the squares of the row vector and the column vector of a matrix or tensor.

[0123] In this example, the F-norm of the symmetric part refers to the square root of the sum of the squares of the row vector and the column vector of the symmetric part of the sparse matrix; the F-norm of the asymmetric part refers to the square root of the sum of the squares of the row vector and the column vector of the asymmetric part of the sparse matrix.

[0124] Maximum ratio of each row (row_ratio): refers to the maximum value of the ratio of the maximum non-zero element in each row divided by the minimum non-zero element in each row when traversing the sparse matrix row by row.

[0125] Maximum ratio of each column (col_ratio): refers to the maximum value of the ratio of the maximum non-zero element in each column divided by the minimum non-zero element in each column when traversing the sparse matrix by column.

[0126] Min_row_std: refers to the standard deviation of the row with the least number of non-zero elements in the sparse matrix.

[0127] The minimum value of the densest row (max_row_min): refers to the minimum value of the row with the largest number of non-zero elements in the sparse matrix.

[0128] The sum of non-zero elements (sum_nnz): refers to the sum of all non-zero elements in the sparse matrix.

[0129] In computer science, time complexity, also known as time complexity, is a function that qualitatively describes the algorithm's running time. It is a function of the length of the string representing the algorithm's input. Time complexity is often expressed using Big O notation, which excludes the function's lower-order terms and the leading coefficient. When expressed this way, time complexity is said to be asymptotic, meaning it considers the case when the input value approaches infinity.

[0130] In this embodiment, the computing device not only extracts the expert features of the sparse matrix, which are usually low-computational features and reflect the local features of the sparse matrix; the computing device also uses multiple resolutions to extract multiple matrix summaries of the sparse matrix. Since the elements included in the matrix summary of the sparse matrix are determined according to multiple sub-matrices determined by the resolution, this means that the matrix summary can describe the global features of the sparse matrix.

[0131] In this way, the computing device optimizes multiple expert features using the matrix summary that describes the global features of the sparse matrix, which helps to improve the uneven quality of expert features and also helps to reduce the feature redundancy of the common features of the sparse matrix. Moreover, when the computing device uses the common features of the sparse matrix for model training, because the feature redundancy of the common features of the sparse matrix is less than or equal to the feature redundancy of the multiple expert features, the amount of feature data required to be processed for model training is reduced, which helps to improve the efficiency of model training.

[0132] In an optional implementation, the computing device may further filter the expert features extracted in S320 to reduce feature redundancy of general features, such as Figure 5 As shown, Figure 5 Schematic diagram of the process of feature extraction method of sparse matrix provided in this application Figure 2 The process of obtaining the general characteristics of the computing device includes the following S331 to S333.

[0133] S331. The computing device screens multiple expert features in at least two ways to obtain at least two feature sets.

[0134] The at least two methods mentioned above include two or three of dependency ensemble extraction, supervised ensemble extraction, and causal analysis, and each method corresponds to a set of feature sets.

[0135] The following is an exemplary description of the expert feature screening method provided in this embodiment.

[0136] 1. Dependency ensemble extraction indicator: Rank the feature importance of multiple expert features to the model and select the feature set according to the ranking gradient. The feature importance indicates the effect of the feature on improving the accuracy of the model.

[0137] Exemplarily, the computing device sorts multiple expert features according to feature importance (the degree of influence / size of influence of a single feature on the accuracy of the model) under different seeds (random seeds), averagely ranks all expert features, divides the feature set (feature set) according to different ranking gradients, inputs the feature set into the model for training, and selects the feature set with the best performance under the dependency ensemble extraction method.

[0138] For example, when the computing device extracts 75 expert features from a sparse matrix, the ranking of each expert feature during model training is calculated, and multiple expert feature complaint models are selected for training according to a certain ranking gradient until a feature set that effectively improves model accuracy is selected. The computing device's test results for the selected feature set are shown in Table 2 below.

[0139] Table 2

[0140]

[0141]

[0142] Among them, selection_feature refers to the number of features filtered out.

[0143] top-1: The recall rate of the top-1 feature, also known as the prediction accuracy of the top-1 feature.

[0144] <1.1count: The ratio of the predicted feature combination to the actual optimal feature combination is less than 110% recall.

[0145] trimmean: The ratio of the predicted feature combination to the actual optimal feature combination is taken as the trimmed mean.

[0146] HL_estimator: The ratio of the predicted feature combination to the actual optimal feature combination is the Hodges-Lehmann estimate.

[0147] Error: Error rate, the number of matrices with incorrect predictions divided by the total number of matrices tested.

[0148] remark: Ranking by gradient, for example, <=60 refers to features with an average ranking below 60.

[0149] 2. Supervised ensemble extraction instructions: Sort the improvement of model accuracy of multiple expert features and select the top N ones, where N is a positive integer.

[0150] Exemplarily, the computing device selects the feature set with the greatest impact (improvement on the model accuracy) under different seeds (random seeds), performs frequency statistics on the selected feature sets, constructs different feature sets based on different frequency gradients as division criteria, inputs the feature sets into the model for training, and selects the feature set with the best performance under the supervised ensemble extraction method.

[0151] For example, the computing device selects 34 expert features with higher frequencies from 75 expert features as a set of feature sets. The test results of the computing device on the selected feature sets are shown in Table 3 below.

[0152] Table 3

[0153] selection_feature top-1 <1.1count trimmean HL_estimator error remark 75 54.29％ 71.57％ 1.062 1.016 11.45％ / 34 55.35％ 71.96％ 1.061 1.015 10.97％ >0 / 10 23 54.96％ 71.66％ 1.062 1.017 10.99％ >5 / 10 22 55.14％ 71.75％ 1.062 1.016 11.06％ >6 / 10 15 45.17％ 61.81％ 1.109 1.027 20.88％ >7 / 10 11 45.38％ 61.99％ 1.1 1.025 20.94％ >8 / 10

[0154] The difference between Table 3 and Table 2 is that the remark in Table 3 is based on frequency. For example, ">5 / 10" indicates a feature that occurs more than 5 times in 10. The meaning of other contents in Table 3 can be found in the description of Table 2 and is not repeated here.

[0155] 3. Causal analysis indication: Select a feature set from multiple expert features that has a positive impact on the model accuracy.

[0156] For example, the computing device uses causal analysis to estimate the impact of the original features: the computing device constructs a dataset, iterates through all expert features, and outputs features that improve model performance under different seeds. Furthermore, the computing device constructs different feature sets based on different frequency gradients, trains the model, and selects the feature set that performs best under causal analysis.

[0157] like Figure 6 As shown, Figure 6 A schematic diagram of the visualization results of a causal analysis provided in this application. Figure 6 "+" in the middle) indicates that the feature has a positive improvement on the model accuracy, and to the right ( Figure 6 "-" in the middle) indicates that the feature has no positive improvement on the model accuracy. Figure 6The features that do not positively improve the model accuracy include: diagonal average (diag_avg), diagonal minimum (diag_min), upper triangular bandwidth (upper_bw), lower triangular bandwidth (lower_bw), fuzzy sparsity (sparity_nnz, the number of non-zero elements with an absolute value less than 1 / total non-zero number), Frobenius norm (F-norm), the proportion of non-zero elements (nnz_frac), the number of non-zero elements (nnz_tot), the number of columns (n_cols), the number of rows (n_rows), etc. Figure 6 Features that positively improve model accuracy include: diagonal standard deviation (diag_std), diagonal maximum (diag_max), number of diagonally dominated rows (diag>row), symmetry measure (sym_det), and whether the sparse matrix is symmetric (symmetry). For descriptions of these features, refer to the general technical descriptions and are not detailed here.

[0158] Please continue to refer to Figure 5 The feature extraction method provided in this embodiment also includes the following S332 and S333.

[0159] S332: The computing device corrects the at least two feature sets obtained in S331 to obtain a candidate universal feature set.

[0160] The candidate universal feature set includes a plurality of candidate universal features.

[0161] Combined with Table 2, Table 3 and Figure 6 From the results, we can see that all three methods find the feature set with the best performance in the middle part of the 75 features, which shows that there is indeed redundancy between the feature sets of different groups, and the optimal feature sets selected by different methods are also different. A single feature screening method cannot guarantee the generalization and universality of the feature set. Therefore, the computing device can perform comprehensive correction on the feature sets selected by the three different methods.

[0162] In one optional example, the comprehensive correction process includes selecting multiple features with high frequency of occurrence from at least two feature sets, and using these multiple features as candidate common feature sets. The features with high frequency of occurrence may be features used in both feature sets, or in all three feature sets. The specific correction process may be determined based on the user's preference for expert feature screening or requirements set in the computing device, and is not limited in this application.

[0163] S333. The computing device processes the candidate universal feature set according to the multiple matrix summaries to obtain universal features of the sparse matrix.

[0164] In this embodiment, in the process of obtaining the universal features of the sparse matrix, the computing device not only uses multiple matrix summaries that characterize the global features of the sparse matrix, but also uses multiple different methods to screen the expert features to obtain a candidate universal feature set. Since the expert features characterize the local features of the sparse matrix, the computing device processes the candidate universal feature set with multiple matrix summaries, which is actually to optimize the screened local features using the global features of the sparse matrix. Therefore, the universal features obtained by the computing device can comprehensively characterize the sparse matrix and reduce the feature redundancy of the universal features.

[0165] In an optional implementation, the training set in the above embodiment includes multiple sparse matrices, and the candidate universal feature set obtained in S332 includes: candidate universal feature sets corresponding to different sparse matrices. With respect to the specific implementation of the above S333, taking the first sparse matrix as any one of the multiple sparse matrices as an example, this embodiment provides a feasible specific example: the computing device processes the candidate universal feature set of the first sparse matrix according to the multiple matrix summaries of the first sparse matrix to obtain the universal features of the first sparse matrix; and the computing device corrects the multiple groups of universal features of the multiple sparse matrices to obtain the universal features of the sparse matrices.

[0166] Optionally, the manner in which the computing device corrects the multiple groups of common features may include: the computing device collaboratively corrects the multiple groups of common features in a union and intersection manner.

[0167] It is worth noting that in the above embodiment, the computing device first obtains the common features of each sparse matrix, and then corrects the common features of multiple sparse matrices, that is, first corrects the expert features of the same sparse matrix, and then corrects the common features of different sparse matrices (dataset correction). However, in other optional examples, after the computing device obtains multiple sets of feature sets after each sparse matrix is screened in multiple ways, the computing device first corrects the multiple sets of feature sets of multiple sparse matrices in one way, and then corrects the feature sets in different ways, that is, first performs data set correction, and then performs expert feature correction.

[0168] In this embodiment, the computing device performs frequency collaborative correction on the feature sets formed by different data sets to form a set of universal features, so that these universal features can be applied to the feature extraction process of sparse matrices including different elements, avoiding the problem of the feature extraction process being unable to adapt due to differences in the elements included in the sparse matrix, and improving the generalization of these universal features.

[0169] Based on the collaborative calibration of different data sets and different expert feature screening methods, the test effect of this embodiment applied to different data sets was tested, as shown below: Figure 7 As shown, Figure 7Schematic diagram of the generalization of the common features provided in this application on different datasets.

[0170] For example, dataset 1 is a mixed-dataset, dataset 2 is SuiteSparse+OpenForm, dataset 3 is SuiteSparse+FreeFEM, and dataset 4 is OpenForm+FreeFEM. For the contents included in SuiteSparse, OpenForm, and FreeFEM, please refer to the description of general technology and will not be repeated here.

[0171] In dataset 1, the recall rate of the general features was 79.53% compared to the performance of the best benchmark (1) within 10% (recommended performance 1.1). In dataset 2, the recall rate of the general features was 76.91% compared to the performance of the best benchmark (1) within 10% (recommended performance 1.1). In dataset 3, the recall rate of the general features was 80.96% compared to the performance of the best benchmark (1) within 10% (recommended performance 1.1). In dataset 4, the recall rate of the general features was 80.27% compared to the performance of the best benchmark (1) within 10% (recommended performance 1.1). It can be seen that the difference between the recommended combinations of the general features determined by the computing device using the above embodiment in four different datasets is small, and the generalization of the general features determined by the above embodiment is high in different datasets.

[0172] During the model training process, the computing device can also estimate the feature extraction time based on information such as the feature extraction time acceptable to the user and the number of features expected to be added.

[0173] Exemplarily, the computing device reduces the time complexity of common features to O(nnz), and adding the number of basic features will determine the overall feature extraction time required, as shown in the following formula (1).

[0174] Total time =a·O(nnz)+b·O(n) Formula (1)

[0175] Among them, Total time is the total feature extraction time, a is the number of features with a time complexity of O(nnz), and b is the number of features with a time complexity of O(n). In actual operation, the computing device allows the user to select the acceptable feature calculation time and the number of features expected to be added, and then gives an approximate calculation time cost.

[0176] In summary, the feature extraction method provided in the embodiment of the present application performs screening based on the expert features of the sparse features extracted by the computing device, and further processes the screened features using the matrix summary of the sparse matrix, thereby determining the universal features of the sparse matrix. Since the expert features are mostly low-computation features and can characterize the local features of the sparse matrix, and multiple matrix summaries indicate the global features of the sparse matrix, the universal features of the sparse matrix are determined by the expert features and the matrix summary, realizing a comprehensive characterization of the sparse matrix in a local feature + global feature manner.

[0177] Furthermore, during the expert feature selection process, the computing device systematically screens expert features using three methods: dependency ensemble extraction, supervised ensemble extraction, and causal analysis. These methods are then expanded across different datasets, improving the generalizability of the general features derived by the computing device. Furthermore, the computing device employs multi-scale pooling to determine multiple matrix summaries of a sparse matrix, which then provide the original information of the sparse matrix. This further refines the information represented by the general features, thereby improving the accuracy and generalization of models trained using these general features.

[0178] It is understood that in order to implement the functions in the above embodiments, the computing device includes hardware structures and / or software modules corresponding to the execution of each function. It should be readily apparent to those skilled in the art that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in hardware or in a manner driven by computer software depends on the specific application scenario and design constraints of the technical solution.

[0179] The above describes in detail the feature extraction method of the sparse matrix provided by this embodiment in conjunction with the accompanying drawings. Figure 8 , describing the sparse matrix feature extraction device provided in this embodiment.

[0180] Figure 8 This is a schematic diagram of the structure of a sparse matrix feature extraction device provided in this application. The feature extraction device 800 can be used to implement the functions of the computing device in the above method embodiment, and thus can also achieve the beneficial effects of the above method embodiment. In this embodiment, the feature extraction device 800 can be as follows Figure 1 The computing device 110 or the computing device 120 shown may also be a computing device provided in subsequent embodiments. It should be understood that the feature extraction device 800 may also be a module (such as a chip) applied to any of the aforementioned computing devices.

[0181] Exemplarily, the feature extraction device 800 includes: an acquisition module 810, an extraction module 820, and a feature processing module 830. The acquisition module 810 is used to acquire a training set, which includes a sparse matrix. The extraction module 820 is used to extract multiple expert features of the sparse matrix and extract multiple matrix summaries of the sparse matrix using multiple resolutions; one resolution corresponds to one matrix summary, and one matrix summary includes multiple elements. The sparse matrix has multiple sub-matrices determined by one resolution, and one element corresponds to one sub-matrix. The feature processing module 830 is used to process multiple expert features based on the multiple matrix summaries to obtain common features of the sparse matrix, where the feature redundancy of the common features of the sparse matrix is less than or equal to the feature redundancy of the multiple expert features.

[0182] The acquisition module 810, extraction module 820, and feature processing module 830, along with other possible modules, can collaboratively implement the various steps of the computing device in the above-described method embodiment. A more detailed description of the acquisition module 810, extraction module 820, and feature processing module 830 can be directly obtained by referring to the relevant description of the computing device in the method embodiment shown in the aforementioned figures, and is not further elaborated here.

[0183] The feature extraction device implements any of the feature extraction methods shown in the aforementioned figures through software. The feature extraction device and its various units may also be software modules. The feature extraction method is implemented by a processor calling the software module. The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0184] Understandably, Figure 8 The device shown is merely an example provided in this embodiment. Depending on the feature extraction process, the feature extraction device may include more or fewer units, and this application is not limited thereto. For example, the feature extraction device may be used as a component in a software acceleration library; when the feature extraction device is used to extract common features of a sparse matrix, the feature extraction device may also be used as a module in model optimization.

[0185] When the feature extraction device is implemented by hardware, the hardware can be implemented by a processor, a chip, or a chip system. The chip system includes one or more chips, each chip including an interface circuit and a control circuit. The interface circuit is used to receive data from other devices outside the chip and transmit it to the control circuit, or send data from the control circuit to other devices outside the chip. The control circuit and the interface circuit are used to implement the method of any possible implementation method in the above embodiments through logic circuits or execution code instructions. The beneficial effects can be found in the description of any aspect of the above embodiments, and will not be repeated here.

[0186] It is understood that the processor in the embodiments of the present application may be a CPU, or other general-purpose processor, digital signal processor (DSP), ASIC, FPGA or other programmable logic device, transistor logic device, hardware component or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0187] in addition, Figure 8 The device shown can also be realized by electronic devices, such as Figure 8 The feature extraction device 800 can be implemented by a controller. For example, the controller includes a control circuit and an interface circuit. The interface circuit is used to send a recognition command and cooperate with the control circuit to execute the feature extraction method of the sparse matrix in any of the above embodiments.

[0188] Figure 8 The feature extraction device in can be realized by a computing device, such as Figure 9 , Figure 9 This is a schematic diagram of the structure of a computing device provided in this application. The computing device 900 can be used to implement the functions of the computing device in the above method embodiment, and thus can also achieve the beneficial effects of the above method embodiment. In this embodiment, the computing device can be Figure 1 The computing device 110 or computing device 120 shown may also be a computing device in subsequent embodiments, or a module (such as a chip) applied to a computing device.

[0189] like Figure 9As shown, computing device 900 may include a processor 920. Optionally, computing device 900 may further include a memory 930 and / or a transceiver 910. Processor 920 is coupled to memory 930 and transceiver 910, for example, by a communication bus. The communication bus may include, but is not limited to, a PCIe bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), and the like.

[0190] The following combination Figure 9 The components of the computing device 900 are described in detail.

[0191] The processor 920 is the control center of the computing device 900 and can be a single processor or a collective term for multiple processing elements. For example, the processor 920 can be one or more CPUs, an ASIC, or one or more integrated circuits configured to implement the embodiments of the present application, such as one or more digital signal processors (DSPs) or one or more field programmable gate arrays (FPGAs).

[0192] Optionally, the processor 920 may execute various functions of the computing device 900 by running or executing software programs stored in the memory 930 and calling data stored in the memory 930. In a specific implementation, as an embodiment, the processor 920 may include one or more CPUs.

[0193] Optionally, the computing device 900 may also include multiple processors. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0194] Wherein, the memory 930 is used to store the software program for executing the solution of the present application, and is controlled by the processor 920 to execute. The specific implementation method can refer to the above method embodiment and will not be repeated here. Exemplarily, the memory 930 can be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 930 can be integrated with the processor 920, or it can exist independently and be accessed through the interface circuit ( Figure 9 (not shown) is coupled to the processor 920, which is not specifically limited in this embodiment of the present application.

[0195] Transceiver 910 is used for communication with other devices. For example, if computing device 900 is a user end (such as a client) or an application server, transceiver 910 can be used to communicate with a storage device or another computing device. For another example, if computing device 900 is a Smart Network Interface Card (SNIC), transceiver 910 can be used to communicate with a storage device or another Smart Network Interface Card. For another example, if computing device 900 is a multi-core chip, transceiver 910 can be used to communicate with another multi-core chip.

[0196] Optionally, the transceiver 910 may include a receiver and a transmitter ( Figure 9 The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function. Optionally, the transceiver 910 can be integrated with the processor 920, or can exist independently and communicate with the processor 920 through the interface circuit ( Figure 9 (not shown) is coupled to the processor 920, which is not specifically limited in this embodiment of the present application.

[0197] In this embodiment, the transceiver 910 is configured to obtain a training set including a sparse matrix. The processor 920 is configured to execute the feature extraction method of any of the aforementioned embodiments in collaboration with the transceiver 910 based on the sparse matrix.

[0198] The method steps in this embodiment can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in RAM, flash memory, ROM, PROM, EPROM, EEPROM, registers, hard disk, mobile hard disk, CD-ROM or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage medium can be located in an ASIC. In addition, the ASIC can be located in a computing device. Of course, the processor and storage medium can also exist as discrete components in a network device or a terminal device.

[0199] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the process or function described in the embodiments of the present application is performed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instruction can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, or a tape; it can also be an optical medium, such as a digital video disc (DVD); it can also be a semiconductor medium, such as a solid state drive (SSD).

[0200] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A feature extraction method for a sparse matrix, characterized in that: The method comprises: Obtaining a training set, wherein the training set includes a sparse matrix; extracting a plurality of expert features of the sparse matrix; Extracting a plurality of matrix summaries of the sparse matrix using a plurality of resolutions; wherein one resolution corresponds to one matrix summary, and the one matrix summary includes a plurality of elements, each element corresponding to a submatrix in the sparse matrix; The plurality of expert features are processed according to the plurality of matrix summaries to obtain a common feature of the sparse matrix, wherein feature redundancy of the common feature of the sparse matrix is less than or equal to feature redundancy of the plurality of expert features.

2. The method according to claim 1, characterized in that The processing of the plurality of expert features according to the plurality of matrix summaries to obtain the common features includes: The plurality of expert features are screened using at least two methods to obtain at least two feature sets; one method corresponds to one feature set, and the at least two methods include two or three of dependency ensemble extraction, supervised ensemble extraction, and causal analysis; Correcting the at least two groups of feature sets to obtain a candidate universal feature set, wherein the candidate universal feature set includes a plurality of candidate universal features; The candidate universal feature set is processed according to the multiple matrix summaries to obtain universal features of the sparse matrix.

3. The method according to claim 2, characterized in that The dependency integration extraction indicates: ranking the feature importance of the multiple expert features to the model, and selecting a feature set according to the ranking gradient, wherein the feature importance indicates the accuracy improvement effect of the feature on the model; The supervised ensemble extraction instruction is: sorting the improvement degree of the model accuracy of the model by the multiple expert features, and selecting the top N ranked features, where N is a positive integer; The causal analysis indicates: selecting a feature set that has a positive improvement on the model accuracy of the model from the multiple expert features.

4. The method according to claim 2 or 3, characterized in that The sparse matrix in the training set includes multiple sparse matrices, and the candidate universal feature set includes: candidate universal feature sets corresponding to different sparse matrices; The step of processing the candidate universal feature set according to the plurality of matrix summaries to obtain universal features of the sparse matrix includes: Processing a candidate common feature set of a first sparse matrix according to a plurality of matrix summaries of the first sparse matrix to obtain common features of the first sparse matrix; the first sparse matrix is any one of the plurality of sparse matrices; Correcting the multiple groups of common features of the multiple sparse matrices to obtain common features of the sparse matrices.

5. The method according to any one of claims 1 to 4, characterized in that An element included in the matrix summary is: an element in the sub-matrix, or a value obtained by processing an element in the sub-matrix.

6. The method according to any one of claims 1 to 5, characterized in that The common features of the sparse matrix include two or more of the following features: the proportion of non-zero elements, the measure of symmetry, the diagonal minimum, the average value of non-zero elements, numerical symmetry, the upper triangle bandwidth, the data symmetry percentage, the average continuous non-zero block, the Frobenius norm of the symmetric part, the Frobenius norm of the asymmetric part, the maximum ratio of each row, the maximum ratio of each column, the standard deviation of the sparsest row, the minimum value of the densest row, the minimum positive value or maximum negative value of the diagonal elements minus the sum of the absolute off-diagonal elements, and the sum of the non-zero elements.

7. The method according to any one of claims 1 to 6, characterized in that The extracting of a plurality of expert features of the sparse matrix comprises: The sparse matrix is input into an expert system to obtain a plurality of expert features of the sparse matrix. The expert system includes a feature extraction layer, and the feature extraction layer is used to extract the expert features of the matrix.

8. A feature extraction device for a sparse matrix, characterized in that: The device comprises: An acquisition module, configured to acquire a training set, wherein the training set includes a sparse matrix; An extraction module is configured to extract a plurality of expert features of the sparse matrix; extract a plurality of matrix summaries of the sparse matrix using a plurality of resolutions; each resolution corresponds to a matrix summary, and each matrix summary includes a plurality of elements, each element corresponding to a submatrix in the sparse matrix; A feature processing module is used to process the multiple expert features according to the multiple matrix summaries to obtain a common feature of the sparse matrix, wherein the feature redundancy of the common feature of the sparse matrix is less than or equal to the feature redundancy of the multiple expert features.

9. The device according to claim 8, characterized in that The feature processing module is specifically configured to: screen the plurality of expert features using at least two methods to obtain at least two feature sets; one method corresponds to one feature set, and the at least two methods include two or three of dependency ensemble extraction, supervised ensemble extraction, and causal analysis; The feature processing module is specifically used to: correct the at least two groups of feature sets to obtain a candidate universal feature set, where the candidate universal feature set includes multiple candidate universal features; and process the candidate universal feature set according to the multiple matrix summaries to obtain the universal features of the sparse matrix.

10. The device according to claim 9, characterized in that The dependency integration extraction indicates: ranking the feature importance of the multiple expert features to the model, and selecting a feature set according to the ranking gradient, wherein the feature importance indicates the accuracy improvement effect of the feature on the model; The supervised ensemble extraction instruction is: sorting the improvement degree of the model accuracy of the model by the multiple expert features, and selecting the top N ranked features, where N is a positive integer; The causal analysis indicates: selecting a feature set that has a positive improvement on the model accuracy of the model from the multiple expert features.

11. The device according to claim 9 or 10, characterized in that The sparse matrix includes a plurality of sparse matrices, and the candidate universal feature set includes: candidate universal feature sets corresponding to different sparse matrices; The feature processing module is specifically used to: process the candidate universal feature set of the first sparse matrix according to multiple matrix summaries of the first sparse matrix to obtain the universal features of the first sparse matrix, where the first sparse matrix is any one of the multiple sparse matrices; and correct multiple groups of universal features of the multiple sparse matrices to obtain the universal features of the sparse matrix.

12. The device according to any one of claims 8 to 11, characterized in that An element included in the matrix summary is: an element in the sub-matrix, or a value obtained by processing an element in the sub-matrix.

13. The device according to any one of claims 8 to 12, characterized in that The common features of the sparse matrix include two or more of the following features: the proportion of non-zero elements, the measure of symmetry, the diagonal minimum, the average value of non-zero elements, numerical symmetry, the upper triangle bandwidth, the data symmetry percentage, the average continuous non-zero block, the Frobenius norm of the symmetric part, the Frobenius norm of the asymmetric part, the maximum ratio of each row, the maximum ratio of each column, the standard deviation of the sparsest row, the minimum value of the densest row, the minimum positive value or maximum negative value of the diagonal elements minus the sum of the absolute off-diagonal elements, and the sum of the non-zero elements.

14. The device according to any one of claims 8 to 13, characterized in that The extraction module is specifically used to: input the sparse matrix into an expert system to obtain multiple expert features of the sparse matrix. The expert system includes a feature extraction layer, and the feature extraction layer is used to extract the expert features of the matrix.

15. A chip, characterized in that: include: Control circuit and interface circuit; The interface circuit is used to obtain a training set and cooperate with the control circuit to execute the method according to any one of claims 1 to 7.

16. A computing device, characterized in that include: processor and memory; The memory is used to store a set of computer instructions. When the processor executes the set of computer instructions, the method according to any one of claims 1 to 7 is performed.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium includes computer instructions; when the computer instructions are executed in a computing device, the computing device executes the method according to any one of claims 1 to 7.

18. A computer program product, characterized in that When the computer program product is run in a computing device, the computing device performs the method according to any one of claims 1 to 7.

Images