Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

394 results about "Weighing matrix" patented technology

In mathematics, a weighing matrix W of order n and weight w is an n × n (0,1,-1)-matrix such that WWᵀ=wIₙ, where Wᵀ is the transpose of W and Iₙ is the identity matrix of order n. For convenience, a weighing matrix of order n and weight w is often denoted by W(n,w). A W(n,n) is a Hadamard matrix and a W(n,n-1) is equivalent to a conference matrix.t

Inference operation acceleration method for processing extremely long text for large language model

The invention discloses a big language model-oriented inference operation acceleration method for processing an extremely long text, which belongs to the technical field of large-scale pre-training language model inference optimization, and specifically comprises the following steps of: selecting a big language model, starting a sparse attention mode, inputting different texts, recording execution time of each attention head and an attention mode type in different layers, and calculating the inference operation speed of the large language model; establishing a statistical database; solving a multi-head attention load balanced distribution strategy of the large language model under the actual text; establishing a weight index table by splitting the weight matrix; retrieving a weight sub-matrix corresponding to each attention head, and loading the weight sub-matrix to a corresponding GPU device; by asynchronously preloading the MHA calculation weight and the MLP calculation weight of the adjacent layers and combining KV cache management, load balancing in the reasoning process is realized. According to the method, dynamic segmentation, recombination and scheduling of the attention head are carried out in the reasoning stage, so that the problems of multi-GPU load imbalance and resource idling are effectively avoided, and the system throughput of long sequence processing is remarkably improved.
Owner:UNIV OF ELECTRONICS SCI & TECH OF CHINA

Strategy question-answering system and method based on vertical domain large model

The invention discloses a strategy question-answering system and method based on a vertical domain large model, and the method comprises the steps: S1, receiving the natural language input of a user, and retrieving an open source data set to obtain an original data set with the comprehensive similarity meeting the requirements; s2, performing necessary data cleaning and preprocessing on the original data set to obtain a training set; s3, performing fine tuning on the pre-processed basic large language model by adopting the training data set, and introducing a low-rank structure to modify a weight matrix to obtain a final model; s4, the problem enters a final model for post-processing to generate prediction output; the system takes a large language basic model subjected to fine tuning training as a core, introduces a low-rank structure to perform fine tuning on model parameters, reduces parameter quantity required by training while keeping model performance, reduces model complexity, reduces an overfitting risk, improves generalization ability, supports deep fusion of a specific field knowledge base and universal field data, and has a wide application prospect. And the accuracy and correlation of questions and answers are improved.
Owner:ENG UNIV OF THE CHINESE PEOPLES ARMED POLICE FORCE

Aircraft part production quality optimization method based on data analysis

The invention relates to the technical field of aircraft manufacturing, and discloses an aircraft part production quality optimization method based on data analysis. The method comprises the following steps: collecting multi-process real-time processing parameters and quality inspection data of a production line, and generating a dynamic quality weight matrix according to a process parameter coupling degree; extracting process path difference characteristics of qualified products and unqualified products in historical batches, and encoding the process path difference characteristics as a quality evolution chain matched with adjacent process parameter mutation relevance; optimizing process parameters at a quality evaluation node, driving a parameter combination to iterate to minimize quality fluctuation, generating an adjustment amount, updating an evolution chain constraint coefficient, and synchronously constructing a stability evaluation function of a correlation weight matrix and a defect propagation path; triggering process compensation according to an adjustment amount gradient, and verifying the cohesion through a tolerance rule by taking a reference parameter matched with the target evolution chain as compensation data; and generating a feedback matrix by using the quality fluctuation index and the compensation result, and correcting the mapping relation between the weight matrix and the evolution chain.
Owner:CHENGDU SEN BO PRECISION MASCH CO LTD

Large language model weight and activation combined quantification method and system

The invention discloses a large language model weight and activation combined quantification method and system, and belongs to the technical field of model quantification. The method comprises the following steps: collecting and preprocessing a calibration set, inputting a large language model to execute forward propagation, and recording an activation matrix; for each embedding dimension, counting maximum activation absolute values of all lexical elements on the dimension; generating a global threshold value by combining a quantile statistical method with the global sensitivity coefficient, and determining that the dimension with the maximum activation absolute value exceeding the global threshold value in the dimensions is an outlier dimension; respectively designing scaling factors for the normal dimension and the outlier dimension, and generating a reconstruction weight matrix; using Bayesian-gradient joint optimization to reconstruct a truncation threshold value of the weight; calculating a scaling factor of the reconstructed weight matrix to obtain a reconstructed quantized weight matrix; quantizing the activation matrix of the current layer according to the embedding dimension by applying a scaling factor; and performing multiplication calculation with the reconstructed quantization weight matrix to obtain a multiplication output result in an integer domain, and then performing unified inverse quantization recovery.
Owner:NANJING UNIV OF POSTS & TELECOMM

MPC-based AGV adaptive path tracking method

The invention relates to an automatic guided vehicle (AGV) adaptive path tracking method based on MPC. The method comprises the following steps: S1, establishing a kinematic discretization error model based on the kinematic characteristics of the two-wheel differential AGV, processing a continuous kinematic equation by adopting an Euler discretization method, expressing a dynamic change relationship between a transverse deviation distance and an angle deviation in a state-space equation form, and generating a kinematic discrete state-space model; the method has the advantages that the continuous equation is processed by establishing the kinematics discretization error model and adopting the Euler discretization method, the deviation dynamic relation is expressed through the state space, the calculation process is simplified, the precision is kept, sensor data are fused, the wheel type odometer, IMU and laser data are integrated through the extended Kalman filtering algorithm, and the precision is improved. The real-time position and angle deviation are calculated, the positioning accuracy is improved, a model prediction controller objective function is designed, a weight matrix and boundary constraint are introduced according to constraint conditions, and the effect of obstacle avoidance constraint is combined.
Owner:SUZHOU AITEN INTELLIGENT TECH CO LTD

Attitude and orbit combined control method for pursuit game of double spacecrafts

The invention discloses an attitude and orbit combined control method for a pursuit game of double spacecrafts, the double spacecrafts comprise a tracking spacecraft and a target spacecraft, and the method comprises the following steps: S1, aiming at an orbit kinetic equation, constructing a target function of the tracking spacecraft; s2, aiming at the weight matrix of the target function in the step 1, estimating Q and R matrixes in the target function by adopting an unscented particle filtering method, and solving a Nash equilibrium solution of pursuit two parties under incomplete information; s3, calculating a Nash equilibrium strategy of the pursuit problem of the double spacecrafts under incomplete information based on a weight matrix obtained by estimating the target function through unscented particle filtering; and S4, when the distance between the pursuit and the escape is smaller than or equal to a set threshold value, the tracking spacecraft begins to carry out attitude adjustment, the target spacecraft is predicted and controlled by using model prediction control, and the attitude of the tracking spacecraft is adjusted according to prediction and control results. According to the method, the autonomous decision-making capability of the spacecraft in on-orbit operation can be improved.
Owner:NAT INNOVATION INST OF DEFENSE TECH PLA ACAD OF MILITARY SCI

Input data sharing and cache optimization method and system in matrix multiplication calculation and application

The invention discloses an input data sharing and cache optimization method in matrix multiplication calculation. The method comprises the steps of 1, segmenting and distributing an input data matrix and a weight matrix according to the number N of calculation cores on a chip; 2, sequentially connecting the plurality of calculation cores end to end to form a data transmission annular structure; step 3, calculating the distributed matrix multiplication by each calculation core, and transmitting the current input sub-matrix of the calculation core to the next calculation core; step 4, performing matrix multiplication operation on the transmitted input sub-matrix and the weight sub-matrix in the next calculation kernel; and 5, iterating transmission and calculation of the input sub-matrixes, and carrying out N rounds of matrix multiplication of the input sub-matrixes and the weight sub-matrixes to complete the whole operation process. The invention further discloses a system for implementing the method, and the system has wide application value.
Owner:SHANGHAI QUSU CHAOWEI TECHNOLOGY CO LTD

Model compression method based on quantized large language model

The invention particularly discloses a model compression method based on a quantized large language model, and relates to the technical field of large language model compression. The method comprises the following steps: S1, performing quantization processing on a pre-trained large language model to obtain a quantized weight matrix, and calculating a pruning optimization objective function after quantization; s2, designing a post-quantization pruning measurement index based on the quantized weight matrix, and combining the pruning measurement index with an absolute value of the quantized weight and a pruning measurement regularization result of the original weight; and S3, based on a pruning measurement index, carrying out importance sorting on the quantized weight, generating a dynamic binary pruning mask, and minimizing quantization and pruning errors through a block optimization algorithm. The method adopts a block quantization post-pruning algorithm, minimizes quantization and pruning errors, and maximizes the model compression ratio on the premise of ensuring the model performance so as to meet the requirements of actual application on the model.
Owner:SHENZHEN UNIV

Collaborative filtering recommendation method and system based on logic rule reasoning

The invention discloses a collaborative filtering recommendation method and system based on logic rule reasoning, and the method comprises the steps: obtaining the original attribute characteristics of a user and a project or the negative characteristics of the original attribute characteristics based on the binary values of the continuous attributes and discrete attributes of the user and the project; performing logic conjunct or logic disjunction on the original attribute features or the negative features to obtain a logic rule matrix of a user-project pair; fusing the attribute feature information of the user and the item to obtain a unique rule weight matrix associated with the user-item pair; obtaining an interaction prediction rule factor based on the logic rule matrix and the rule weight matrix; and performing model training and prediction by using the interactive prediction rule factor. Through fusion of a neural symbol reasoning technology and a complete logic operator, dynamic mining of a logic rule between a user attribute and a project attribute is realized.
Owner:JIANGSU YEYOO E-CLOUD SOFTWARE CO LTD

Dynamic low-rank estimation for transformer-based language models

A method includes accessing a plurality of weight matrices of a machine learning model. The method also includes, for each weight matrix, decomposing the weight matrix into a U matrix, an S matrix, and a V matrix using singular value decomposition. The S matrix is a diagonal matrix, and a singular group corresponds to each element in the S matrix. The method further includes, for each weight matrix, determining an importance score of each singular group. The importance score of the singular group represents a change in loss if the singular group is removed from the machine learning model. The method also includes, for each weight matrix, ranking the singular groups across the plurality of weight matrices based on the importance scores. In addition, the method includes, for each weight matrix, identifying one or more of the singular groups to prune based on the ranking of the singular groups.
Owner:SAMSUNG ELECTRONICS CO LTD

Quantization method, reasoning method and equipment of industry large model and storage medium

The embodiment of the invention provides a quantification method and reasoning method of an industry large model, equipment and a storage medium. When an original weight matrix of a network layer in an industry large model is quantified, values of matrix elements on diagonals of an original Hessian matrix are obtained based on an input data matrix of the network layer and can represent activation values, and the positions of columns in the original weight matrix and the original Hessian matrix of the network layer are reordered according to an activation value descending mode. Carrying out matrix partitioning processing on the reordered first weight matrix and the first Hessian matrix respectively, and quantizing the first sub-weight matrix based on the quantization parameter of each sub-weight matrix in the first weight matrix in sequence by taking the partitioned matrix as granularity; the method comprises the steps of quantizing a first weight matrix, updating other first weight sub-matrixes which are not quantized based on the quantized first weight sub-matrix, and finally determining a quantization result of an original weight matrix of a network layer based on a quantization result of each first weight sub-matrix in the first weight matrix, so that the size of an industry large model is compressed.
Owner:HANGZHOU ALICLOUD FEITIAN INFORMATION TECH CO LTD

Large-model low-rank adaptive layer weight matrix fine tuning method

The invention belongs to the technical field of large models of deep learning, and provides a large-model low-rank adaptive layer weight matrix fine tuning method in order to reduce calculation complexity in a low-rank weight matrix multiplication process in a low-rank adaptive method and optimize calculation power and memory requirements required by a training model. The two low-rank weight matrixes are copied, spliced and then added, or the two low-rank weight matrixes are divided into the sub-matrixes and then the sub-matrixes are added one by one, so that compared with a mode of multiplying the two low-rank weight matrixes, the calculation complexity of matrix low-rank decomposition can be reduced, meanwhile, the rank of the low-rank weight matrixes can be improved, and the calculation efficiency of matrix low-rank decomposition is improved. When different downstream tasks are finely adjusted, the corresponding weight parameters are independently trained and switched, so that the training efficiency of different fine adjustment weights is improved, and the weight memory requirement is reduced.
Owner:JILIN PROVINCE JILIN XIANGYUN INFORMATION TECH CO LTD

Matrix decomposition and reconstruction method and system for large model reasoning scene and application

The invention discloses a matrix decomposition and reconstruction method for a large model reasoning scene, and the method comprises the following steps: 1, carrying out the error statistics of conventional nodes of a network layer in a large model, and screening weight nodes capable of carrying out low-rank decomposition through a sensitivity analysis strategy; step 2, gradually performing low-rank setting, decomposing the weight matrixes in the weight nodes which can be subjected to low-rank decomposition and are obtained by screening in the step 1, and obtaining and recording respective optimal low-rank values of the decomposable weight matrixes; and step 3, performing network reconstruction on the original model based on the decomposable weight matrix and the corresponding low-rank value. The invention further discloses a system for implementing the method and a corresponding application, and the system and the application have a wide application scene.
Owner:SHANGHAI QUSU CHAOWEI TECHNOLOGY CO LTD

Federal training fine tuning method and device based on low-rank approximate quantization

The invention provides a federated training fine tuning method and device based on low-rank approximate quantization, and the method comprises the steps: carrying out the multi-round alternative optimization of an original weight matrix corresponding to an initial local generation type language model of a current iteration training round based on quantization and low-rank approximate processing, obtaining a target integer weight matrix corresponding to the original weight matrix and a target low-rank approximation matrix; fine-tuning the target low-rank approximation matrix by using local data to obtain a trained low-rank approximation matrix corresponding to the current iteration training round, and taking the target integer weight matrix and the trained low-rank approximation matrix as locally quantized generative language model parameters corresponding to the current iteration training round, and sending the locally quantized generative language model parameters to aggregation end equipment in a federated learning system where the local quantized generative language model parameters are located so as to perform global model aggregation. According to the method and the device, the problem that equipment computing resource consumption, computing precision, communication resource occupancy rate and privacy protection are difficult to balance in a federated learning scene of a current generative language model can be solved.
Owner:BEIJING UNIV OF POSTS & TELECOMM

Deep learning-based belief propagation LDPC decoding method and device, medium and program product

The invention provides a belief propagation LDPC decoding method and device based on deep learning, a medium and a program product. The method comprises the following steps: acquiring an LDPC code training sample set corresponding to a check matrix H; according to the check matrix H, determining a weight matrix and an activation function of each layer of a deep learning decoding model; based on the weight matrix and the activation function, using a belief propagation algorithm to establish a deep learning decoding model; training the deep learning decoding model by using the training sample set, and storing the trained deep learning decoding model as a decoder; and sending the LDPC code soft information which is corresponding to the check matrix H and needs to be decoded into the decoder to complete LDPC decoding. According to the method, iterative operation is completed by using deep learning in the error correction decoding process, the number of decoding iterations is reduced, and the sequence sent by the sending end is restored from the sequence containing noise and interference, so that the problem of decoding error codes caused by noise and other interference factors is solved.
Owner:10TH RES INST OF CETC

Fault analysis method for three-phase intelligent electric energy meter

The invention relates to the field of data processing, in particular to a fault analysis method for a three-phase intelligent electric energy meter, which comprises the following steps of: acquiring operation data of the three-phase intelligent electric energy meter in real time, and constructing a three-phase data sequence according to an acquisition time sequence; calculating correlation values between a target sequence and other sequences at a target moment by using a DTW algorithm, calculating respective weights of the three-phase sequences at the target moment based on all correlation values corresponding to the target moment, and constructing a collaborative weight matrix between the target moment and a reference moment according to all weights corresponding to the target moment and the reference moment; an existing Euclidean distance is improved by using the collaborative weight matrix, and an LOF value of data at each moment is acquired by using the improved Euclidean distance in combination with an LOF algorithm; and when the LOF value is greater than a preset LOF abnormal threshold value, judging that the electric energy meter has a fault at the moment. According to the method, the Euclidean distance is improved, the distance between the abnormal data point and the normal point is increased, the abnormal point is highlighted, and the accuracy of power consumption data anomaly detection is improved.
Owner:YOONO ENERGY TECH (JIANGSU) CO LTD

Fine adjustment method and device for large language model

The embodiment of the invention provides a fine tuning method and device for a large language model. The method comprises the steps that a weight matrix of a pre-trained large language model is acquired, the weight matrix is decomposed into an amplitude vector and a first direction matrix, a first low-rank matrix and a second low-rank matrix are initialized, and the product of the first low-rank matrix and the second low-rank matrix is used for fitting an increment matrix of the first direction matrix; the first direction matrix is kept unchanged, multiple rounds of fine tuning are executed, and each round of fine tuning comprises the steps that a training text is input into a large language model, and calculation of a loss function is conducted; updating the amplitude vector based on the loss function; determining a first gradient matrix of the loss function relative to the current direction matrix; obtaining an updated first low-rank matrix and an updated second low-rank matrix according to the first gradient matrix; and determining the increment matrix of the round based on the product of the updated first low-rank matrix and the second low-rank matrix, and superposing the increment matrix of the round on the first direction matrix to serve as the updated direction matrix of the round.
Owner:ALIPAY (HANGZHOU) INFORMATION TECH CO LTD

Calculation device, calculation method, and program

The invention relates to a calculation device, a calculation method, and a program. This computing device (10) computes a matrix product of a sparse matrix and a weight matrix in a neural network, the sparse matrix having a structure in which a predetermined number of non-zero elements are included in each block of a predetermined size. The sparse matrix is defined by a local index representing the position of the non-zero element in each block and the value of the non-zero element, and the arithmetic device (10) performs: a process for acquiring, from the weight matrix, an element corresponding to the position of the non-zero element in the sparse matrix by referring to the local index; and multiplication and accumulation operation of non-zero elements of the sparse matrix and corresponding elements of the weight matrix is carried out.
Owner:DENSO CORP

Method and device for compressing large language model through singular value decomposition based on gradient attribution

The invention relates to the technical field of large model compression, and discloses a method, a device, equipment and a medium for compressing a large language model through singular value decomposition based on gradient attribution, and the method comprises the steps: calculating the gradient of an original weight matrix of each linear layer on an auxiliary calibration data set, during singular value decomposition of the original weight matrix, sparseness is distributed according to the gradient of the weight of each layer; collecting the gradient of each singular value when the original weight matrix passes through singular value decomposition on the auxiliary calibration data set; using an approximate loss item to represent the importance of each singular value when the original weight matrix passes through singular value decomposition, and filtering unimportant singular values according to a preset dynamic threshold value; and setting the filtered singular value as an index of the original weight matrix when the original weight matrix passes through singular value decomposition, and performing low-rank decomposition on the singular value related to the index in the original weight matrix layer by layer according to a gradient descent method of each singular value until the compression ratio of the large language model is met. And the compression ratio of the model can be obviously improved. The method is applied to application scenes such as financial science and technology large model compression.
Owner:PING AN TECH (SHENZHEN) CO LTD

Apparatus, data structure and method for adjusting weights of neural networks of models

Apparatus, data structure and method of adjusting weights of a neural network of a model, processing inputs of the model and outputting outputs of the model, in which the method comprises: providing training data, the training data comprising the inputs of the model and reference truth values for the outputs of the model, the reference truth values corresponding to the inputs of the model in the training data; providing an adjustment method set for adjusting the weight; determining principal component decomposition of a weight matrix including weights; determining a feature value of a covariance matrix corresponding to the feature vector; rearranging the eigenvectors in the matrix in an order that produces a monotonically decreasing order of eigenvalues associated with the eigenvectors; rearranging the weights in the weight matrix according to the rearrangement sequence of the feature vectors in the matrix; dividing the weight matrix into groups of weights; associating at least one of the groups with an adjustment method selected from the set of adjustment methods; and adjusting the weights in the at least one group on the training data using an adjustment method.
Owner:ROBERT BOSCH GMBH

Inference model compression method and apparatus

Disclosed are an inference model compression method and apparatus, which relate to the technical field of computers. The method comprises: inputting inference data into an inference model, and executing the inference model; then, for a first network layer in the inference model, acquiring a weight matrix of the first network layer and input data of the first network layer, and calculating the product of the weight matrix of the first network layer and the input data of the first network layer to obtain a result matrix that indicates the importance of weight values in the weight matrix; and next, in descending order of the importance, selecting the weight values that rank in the last a% in the weight matrix of the first network layer to perform a sparsification operation, and executing the sparsification operation on a plurality of network layers in the inference model to obtain a sparsified inference model. The first network layer is any one layer among the plurality of network layers, the result matrix and the weight matrix have the same size, and the magnitude of the value at the same position in the result matrix as in the weight matrix indicates the importance of the weight value at the same position in the weight matrix.
Owner:HUAWEI TECH CO LTD

Multi-label text classification method based on positive and negative label learning and label correlation

The invention relates to a multi-label text classification method based on positive and negative label learning and label correlation, and belongs to the field of multi-label classification. Comprising the following steps: constructing a feedforward neural network model with double hidden layers; initializing a model component; reading features and label information of samples in the training set, and generating a feature matrix and a label matrix; randomly initializing a weight matrix and an offset parameter of the model; inputting the feature matrix into an input layer of the model, and calculating neuron output layer by layer; calculating gradients of weight matrixes and bias parameters among layers in the model by adopting a composite error function, dynamically adjusting the gradients by utilizing an Adam optimization algorithm, and updating the weight matrixes and the bias parameters; when the error change amplitude is lower than a threshold value or reaches a preset number of iterations, stopping training; and after model convergence, predicting the test set to form a final multi-label classification result. According to the method, the limitation of traditional text classification is broken through through a deep learning technology, and high-precision and high-efficiency classification of complex text data is realized.
Owner:KUNMING UNIV OF SCI & TECH

Systems and methods of data processing for maching learning

Provided are systems, methods, and apparatuses of data processing for machine learning. In one or more examples, the systems, devices, and methods include determining priority values for elements of a weight matrix based on a gradient of a loss function of an AI model and the weight matrix; determining an index value based on a number of elements in the weight importance matrix and a sparsity ratio; determining a threshold based on sorting the elements of the weight importance matrix in sequential order and determining a value of an element of the sorted weight importance matrix based on using the index value as an index of the sorted weight importance matrix; determining a pruned weights matrix based on the threshold; and processing a query using an updated AI model, the updated AI model being based on the pruned weights matrix being implemented in the AI model.
Owner:SAMSUNG ELECTRONICS CO LTD

Improvement method of large language model, electronic equipment and storage medium

The invention discloses an improvement method of a large language model, electronic equipment and a storage medium, and the method comprises the steps: calculating a correlation matrix between middle layers through a public data set, and calculating mutual information redundancy between the layers based on the correlation matrix between the middle layers, an inter-layer redundancy matrix is constructed through mutual information redundancy among the layers; calculating the total redundancy of each layer based on the redundancy matrix, and generating a layer importance factor through function conversion; establishing an objective function based on the layer importance factors and constructing a linear optimization model of sparse rate distribution of each layer; solving the optimization model by adopting a linear programming algorithm to obtain an optimal sparse rate distribution scheme of each layer; differentiated pruning is carried out on each layer of weight matrix according to the distribution scheme, a weight parameter with the maximum amplitude in each layer of weight matrix is reserved, the rest parameters are set to be zero, and a sparse weight matrix is obtained; according to the method, the features can be selectively pruned in the middle layer, so that fine-grained optimization of a large language model is realized.
Owner:AISPEECH CO LTD

Multi-frequency multi-system combined pseudo-range observation value calculation method and system

The invention discloses a multi-frequency multi-system combined pseudo-range observation value calculation method and system, and relates to the technical field of information, and the method comprises the steps: obtaining satellite observation data received by a GNSS receiver, constructing a diagonal weight matrix, and carrying out the calculation to obtain a preliminary multi-frequency combined model coefficient estimation matrix; calculating standardized residual identification of the observation value, correcting an abnormal pseudo-range observation value, and iteratively correcting and updating the weight matrix to obtain an updated multi-frequency combination model coefficient estimation matrix; and judging whether the multi-frequency combination model coefficient estimation matrix meets a cut-off requirement or not to obtain an optimal multi-frequency combination model coefficient matrix calculation combination pseudo-range observation value. The method can better adapt to the change of observation data, improves the accuracy of model estimation, reduces the positioning error, and enables the positioning result to be more reliable. Unnecessary iterative computation is avoided, computing resources are saved, the accuracy and reliability of a final model coefficient matrix are ensured, and the overall performance of the system is improved.
Owner:GUIZHOU POWER GRID CO LTD

Memory mapping method and system for rotary position embedding operation

Provided are a memory mapping method and system for a rotary position embedding operation. The memory mapping system according to one embodiment may include: a preprocessing unit for adjusting the positions of the values in a weight matrix for a pretrained artificial intelligence model such that the values of the elements required in a rotary position embedding operation, which is the next operation after a matrix multiplication operation, are adjacent to one another; a weight memory for storing the weight matrix with the positions of the values adjusted; and a hardware operator for generating a query vector or a key vector by processing the matrix multiplication operation of an input vector and the weight matrix and for processing the rotary position embedding operation of the query vector or the key vector by using the values of the elements adjacent to one another.
Owner:HYPERACCEL CO LTD

Deep learning eigen-beamforming

According to one example embodiment, a method is provided including initializing a first recurrent neural network with a first vector; iteratively training a weight matrix for one or more layers of the first recurrent neural network, wherein W(t) is the weight matrix of the layer of the first recurrent neural network corresponding to iteration t, and wherein W(t) is based at least on a channel covariance matrix; the first vector; and an output from a previous layer of the first recurrent neural network corresponding to iteration, t−1: and determining a first eigenvector based on a converged output from the first recurrent neural network, wherein the first eigenvector is used to perform beamforming for multiple input multiple output reception and / or transmission.
Owner:NOKIA SOLUTIONS & NETWORKS OY

Clustering method and device for graph regular non-negative matrix factorization and medium

The invention belongs to the technical field of image classification, and discloses a clustering method and device for graph regular non-negative matrix factorization and a medium, and the method comprises the steps: obtaining matrix information data and a to-be-clustered number; defining four initialized sparse matrixes, respectively and correspondingly constructing a matrix E, a weight matrix U, a coefficient matrix V and an assistance matrix Z, and setting an initial hyper-parameter value mu; determining a target equation and the maximum number of iterations, and iteratively solving a construction matrix E, a weight matrix U, a coefficient matrix V, an assistance matrix Z, a Lagrange operator Y1 and a Lagrange operator Y2 according to an updating formula; and outputting a final coefficient matrix V under the condition that the number of iterations is maximum or iteration converges, and obtaining a clustering result. According to the method, the target function construction method is improved, and better constraint conditions are added to capture deep information features, so that the robustness and accuracy of image classification are improved, and a better clustering effect is obtained.
Owner:湖南工商大学

Feature selection method and data reconstruction method for shoreline image feature selection

The invention discloses a feature selection method and a data reconstruction method for shoreline image feature selection. The method comprises the following steps: S1, integrating original data into a sample data set matrix X; s2, initializing a reconstruction transformation matrix W, a transformation matrix Q, an eigenvalue regression coefficient matrix A, a coordinate basis matrix B, a coding matrix E, an auxiliary matrix F and an auxiliary matrix D as unit matrixes, initializing a weight vector p and a mean vector v of a data set as unit vectors, and initializing a weight matrix P = diag (p); and S3, updating the coding matrix E, the auxiliary matrix F and the like based on the data set matrix X, the mean vector v of the data set, the transformation matrix Q, the eigenvalue regression coefficient matrix A, the coordinate basis matrix B and the weight matrix P. According to the invention, the accuracy of data reconstruction and the validity of feature selection can be improved.
Owner:THIRD INSTITUTE OF OCEANOGRAPHY STATE OCEANI C ADMINISTRATION

Explainable sequence recommendation method, system, and apparatus that fuse concepts and behaviors

The application provides an explainable sequence recommendation method, system and device fusing concepts and behaviors, relates to the technical field of recommendation systems, and the method comprises the following steps: obtaining a concept library and a historical behavior sequence of a user; obtaining a historical concept sequence and a concept embedding matrix according to the historical behavior sequence and the concept library; obtaining a concept weight matrix and a concept behavior sequence set according to the historical behavior sequence, the historical concept sequence and the concept embedding matrix in combination with an association strategy of concepts and behaviors; and performing sequence recommendation on the user according to the historical behavior sequence, the concept embedding matrix, the concept weight matrix and the concept behavior sequence set to obtain an explainable sequence recommendation result. By introducing the association strategy of concepts and behaviors, the application not only effectively fuses abstract concept information and specific behavior information, but also dynamically adjusts the understanding and explanation of user preferences through the concept embedding matrix and the concept weight matrix, thereby improving the accuracy and explainability of the sequence recommendation result.
Owner:SHANTOU UNIV