279 results about "Weighing matrix" patented technology

The invention discloses a strategy question-answering system and method based on a vertical domain large model, and the method comprises the steps: S1, receiving the natural language input of a user, and retrieving an open source data set to obtain an original data set with the comprehensive similarity meeting the requirements; s2, performing necessary data cleaning and preprocessing on the original data set to obtain a training set; s3, performing fine tuning on the pre-processed basic large language model by adopting the training data set, and introducing a low-rank structure to modify a weight matrix to obtain a final model; s4, the problem enters a final model for post-processing to generate prediction output; the system takes a large language basic model subjected to fine tuning training as a core, introduces a low-rank structure to perform fine tuning on model parameters, reduces parameter quantity required by training while keeping model performance, reduces model complexity, reduces an overfitting risk, improves generalization ability, supports deep fusion of a specific field knowledge base and universal field data, and has a wide application prospect. And the accuracy and correlation of questions and answers are improved.

The invention relates to the technical field of aircraft manufacturing, and discloses an aircraft part production quality optimization method based on data analysis. The method comprises the following steps: collecting multi-process real-time processing parameters and quality inspection data of a production line, and generating a dynamic quality weight matrix according to a process parameter coupling degree; extracting process path difference characteristics of qualified products and unqualified products in historical batches, and encoding the process path difference characteristics as a quality evolution chain matched with adjacent process parameter mutation relevance; optimizing process parameters at a quality evaluation node, driving a parameter combination to iterate to minimize quality fluctuation, generating an adjustment amount, updating an evolution chain constraint coefficient, and synchronously constructing a stability evaluation function of a correlation weight matrix and a defect propagation path; triggering process compensation according to an adjustment amount gradient, and verifying the cohesion through a tolerance rule by taking a reference parameter matched with the target evolution chain as compensation data; and generating a feedback matrix by using the quality fluctuation index and the compensation result, and correcting the mapping relation between the weight matrix and the evolution chain.

The invention discloses a large language model weight and activation combined quantification method and system, and belongs to the technical field of model quantification. The method comprises the following steps: collecting and preprocessing a calibration set, inputting a large language model to execute forward propagation, and recording an activation matrix; for each embedding dimension, counting maximum activation absolute values of all lexical elements on the dimension; generating a global threshold value by combining a quantile statistical method with the global sensitivity coefficient, and determining that the dimension with the maximum activation absolute value exceeding the global threshold value in the dimensions is an outlier dimension; respectively designing scaling factors for the normal dimension and the outlier dimension, and generating a reconstruction weight matrix; using Bayesian-gradient joint optimization to reconstruct a truncation threshold value of the weight; calculating a scaling factor of the reconstructed weight matrix to obtain a reconstructed quantized weight matrix; quantizing the activation matrix of the current layer according to the embedding dimension by applying a scaling factor; and performing multiplication calculation with the reconstructed quantization weight matrix to obtain a multiplication output result in an integer domain, and then performing unified inverse quantization recovery.

The invention relates to an automatic guided vehicle (AGV) adaptive path tracking method based on MPC. The method comprises the following steps: S1, establishing a kinematic discretization error model based on the kinematic characteristics of the two-wheel differential AGV, processing a continuous kinematic equation by adopting an Euler discretization method, expressing a dynamic change relationship between a transverse deviation distance and an angle deviation in a state-space equation form, and generating a kinematic discrete state-space model; the method has the advantages that the continuous equation is processed by establishing the kinematics discretization error model and adopting the Euler discretization method, the deviation dynamic relation is expressed through the state space, the calculation process is simplified, the precision is kept, sensor data are fused, the wheel type odometer, IMU and laser data are integrated through the extended Kalman filtering algorithm, and the precision is improved. The real-time position and angle deviation are calculated, the positioning accuracy is improved, a model prediction controller objective function is designed, a weight matrix and boundary constraint are introduced according to constraint conditions, and the effect of obstacle avoidance constraint is combined.

The invention discloses an input data sharing and cache optimization method in matrix multiplication calculation. The method comprises the steps of 1, segmenting and distributing an input data matrix and a weight matrix according to the number N of calculation cores on a chip; 2, sequentially connecting the plurality of calculation cores end to end to form a data transmission annular structure; step 3, calculating the distributed matrix multiplication by each calculation core, and transmitting the current input sub-matrix of the calculation core to the next calculation core; step 4, performing matrix multiplication operation on the transmitted input sub-matrix and the weight sub-matrix in the next calculation kernel; and 5, iterating transmission and calculation of the input sub-matrixes, and carrying out N rounds of matrix multiplication of the input sub-matrixes and the weight sub-matrixes to complete the whole operation process. The invention further discloses a system for implementing the method, and the system has wide application value.

The invention particularly discloses a model compression method based on a quantized large language model, and relates to the technical field of large language model compression. The method comprises the following steps: S1, performing quantization processing on a pre-trained large language model to obtain a quantized weight matrix, and calculating a pruning optimization objective function after quantization; s2, designing a post-quantization pruning measurement index based on the quantized weight matrix, and combining the pruning measurement index with an absolute value of the quantized weight and a pruning measurement regularization result of the original weight; and S3, based on a pruning measurement index, carrying out importance sorting on the quantized weight, generating a dynamic binary pruning mask, and minimizing quantization and pruning errors through a block optimization algorithm. The method adopts a block quantization post-pruning algorithm, minimizes quantization and pruning errors, and maximizes the model compression ratio on the premise of ensuring the model performance so as to meet the requirements of actual application on the model.

The invention discloses a collaborative filtering recommendation method and system based on logic rule reasoning, and the method comprises the steps: obtaining the original attribute characteristics of a user and a project or the negative characteristics of the original attribute characteristics based on the binary values of the continuous attributes and discrete attributes of the user and the project; performing logic conjunct or logic disjunction on the original attribute features or the negative features to obtain a logic rule matrix of a user-project pair; fusing the attribute feature information of the user and the item to obtain a unique rule weight matrix associated with the user-item pair; obtaining an interaction prediction rule factor based on the logic rule matrix and the rule weight matrix; and performing model training and prediction by using the interactive prediction rule factor. Through fusion of a neural symbol reasoning technology and a complete logic operator, dynamic mining of a logic rule between a user attribute and a project attribute is realized.

The embodiment of the invention provides a quantification method and reasoning method of an industry large model, equipment and a storage medium. When an original weight matrix of a network layer in an industry large model is quantified, values of matrix elements on diagonals of an original Hessian matrix are obtained based on an input data matrix of the network layer and can represent activation values, and the positions of columns in the original weight matrix and the original Hessian matrix of the network layer are reordered according to an activation value descending mode. Carrying out matrix partitioning processing on the reordered first weight matrix and the first Hessian matrix respectively, and quantizing the first sub-weight matrix based on the quantization parameter of each sub-weight matrix in the first weight matrix in sequence by taking the partitioned matrix as granularity; the method comprises the steps of quantizing a first weight matrix, updating other first weight sub-matrixes which are not quantized based on the quantized first weight sub-matrix, and finally determining a quantization result of an original weight matrix of a network layer based on a quantization result of each first weight sub-matrix in the first weight matrix, so that the size of an industry large model is compressed.

The invention provides a federated training fine tuning method and device based on low-rank approximate quantization, and the method comprises the steps: carrying out the multi-round alternative optimization of an original weight matrix corresponding to an initial local generation type language model of a current iteration training round based on quantization and low-rank approximate processing, obtaining a target integer weight matrix corresponding to the original weight matrix and a target low-rank approximation matrix; fine-tuning the target low-rank approximation matrix by using local data to obtain a trained low-rank approximation matrix corresponding to the current iteration training round, and taking the target integer weight matrix and the trained low-rank approximation matrix as locally quantized generative language model parameters corresponding to the current iteration training round, and sending the locally quantized generative language model parameters to aggregation end equipment in a federated learning system where the local quantized generative language model parameters are located so as to perform global model aggregation. According to the method and the device, the problem that equipment computing resource consumption, computing precision, communication resource occupancy rate and privacy protection are difficult to balance in a federated learning scene of a current generative language model can be solved.

The invention provides a belief propagation LDPC decoding method and device based on deep learning, a medium and a program product. The method comprises the following steps: acquiring an LDPC code training sample set corresponding to a check matrix H; according to the check matrix H, determining a weight matrix and an activation function of each layer of a deep learning decoding model; based on the weight matrix and the activation function, using a belief propagation algorithm to establish a deep learning decoding model; training the deep learning decoding model by using the training sample set, and storing the trained deep learning decoding model as a decoder; and sending the LDPC code soft information which is corresponding to the check matrix H and needs to be decoded into the decoder to complete LDPC decoding. According to the method, iterative operation is completed by using deep learning in the error correction decoding process, the number of decoding iterations is reduced, and the sequence sent by the sending end is restored from the sequence containing noise and interference, so that the problem of decoding error codes caused by noise and other interference factors is solved.

The invention relates to the field of data processing, in particular to a fault analysis method for a three-phase intelligent electric energy meter, which comprises the following steps of: acquiring operation data of the three-phase intelligent electric energy meter in real time, and constructing a three-phase data sequence according to an acquisition time sequence; calculating correlation values between a target sequence and other sequences at a target moment by using a DTW algorithm, calculating respective weights of the three-phase sequences at the target moment based on all correlation values corresponding to the target moment, and constructing a collaborative weight matrix between the target moment and a reference moment according to all weights corresponding to the target moment and the reference moment; an existing Euclidean distance is improved by using the collaborative weight matrix, and an LOF value of data at each moment is acquired by using the improved Euclidean distance in combination with an LOF algorithm; and when the LOF value is greater than a preset LOF abnormal threshold value, judging that the electric energy meter has a fault at the moment. According to the method, the Euclidean distance is improved, the distance between the abnormal data point and the normal point is increased, the abnormal point is highlighted, and the accuracy of power consumption data anomaly detection is improved.

The invention relates to a calculation device, a calculation method, and a program. This computing device (10) computes a matrix product of a sparse matrix and a weight matrix in a neural network, the sparse matrix having a structure in which a predetermined number of non-zero elements are included in each block of a predetermined size. The sparse matrix is defined by a local index representing the position of the non-zero element in each block and the value of the non-zero element, and the arithmetic device (10) performs: a process for acquiring, from the weight matrix, an element corresponding to the position of the non-zero element in the sparse matrix by referring to the local index; and multiplication and accumulation operation of non-zero elements of the sparse matrix and corresponding elements of the weight matrix is carried out.

Apparatus, data structure and method of adjusting weights of a neural network of a model, processing inputs of the model and outputting outputs of the model, in which the method comprises: providing training data, the training data comprising the inputs of the model and reference truth values for the outputs of the model, the reference truth values corresponding to the inputs of the model in the training data; providing an adjustment method set for adjusting the weight; determining principal component decomposition of a weight matrix including weights; determining a feature value of a covariance matrix corresponding to the feature vector; rearranging the eigenvectors in the matrix in an order that produces a monotonically decreasing order of eigenvalues associated with the eigenvectors; rearranging the weights in the weight matrix according to the rearrangement sequence of the feature vectors in the matrix; dividing the weight matrix into groups of weights; associating at least one of the groups with an adjustment method selected from the set of adjustment methods; and adjusting the weights in the at least one group on the training data using an adjustment method.

Disclosed are an inference model compression method and apparatus, which relate to the technical field of computers. The method comprises: inputting inference data into an inference model, and executing the inference model; then, for a first network layer in the inference model, acquiring a weight matrix of the first network layer and input data of the first network layer, and calculating the product of the weight matrix of the first network layer and the input data of the first network layer to obtain a result matrix that indicates the importance of weight values in the weight matrix; and next, in descending order of the importance, selecting the weight values that rank in the last a% in the weight matrix of the first network layer to perform a sparsification operation, and executing the sparsification operation on a plurality of network layers in the inference model to obtain a sparsified inference model. The first network layer is any one layer among the plurality of network layers, the result matrix and the weight matrix have the same size, and the magnitude of the value at the same position in the result matrix as in the weight matrix indicates the importance of the weight value at the same position in the weight matrix.

The invention relates to a multi-label text classification method based on positive and negative label learning and label correlation, and belongs to the field of multi-label classification. Comprising the following steps: constructing a feedforward neural network model with double hidden layers; initializing a model component; reading features and label information of samples in the training set, and generating a feature matrix and a label matrix; randomly initializing a weight matrix and an offset parameter of the model; inputting the feature matrix into an input layer of the model, and calculating neuron output layer by layer; calculating gradients of weight matrixes and bias parameters among layers in the model by adopting a composite error function, dynamically adjusting the gradients by utilizing an Adam optimization algorithm, and updating the weight matrixes and the bias parameters; when the error change amplitude is lower than a threshold value or reaches a preset number of iterations, stopping training; and after model convergence, predicting the test set to form a final multi-label classification result. According to the method, the limitation of traditional text classification is broken through through a deep learning technology, and high-precision and high-efficiency classification of complex text data is realized.

Provided are systems, methods, and apparatuses of data processing for machine learning. In one or more examples, the systems, devices, and methods include determining priority values for elements of a weight matrix based on a gradient of a loss function of an AI model and the weight matrix; determining an index value based on a number of elements in the weight importance matrix and a sparsity ratio; determining a threshold based on sorting the elements of the weight importance matrix in sequential order and determining a value of an element of the sorted weight importance matrix based on using the index value as an index of the sorted weight importance matrix; determining a pruned weights matrix based on the threshold; and processing a query using an updated AI model, the updated AI model being based on the pruned weights matrix being implemented in the AI model.

The invention discloses an improvement method of a large language model, electronic equipment and a storage medium, and the method comprises the steps: calculating a correlation matrix between middle layers through a public data set, and calculating mutual information redundancy between the layers based on the correlation matrix between the middle layers, an inter-layer redundancy matrix is constructed through mutual information redundancy among the layers; calculating the total redundancy of each layer based on the redundancy matrix, and generating a layer importance factor through function conversion; establishing an objective function based on the layer importance factors and constructing a linear optimization model of sparse rate distribution of each layer; solving the optimization model by adopting a linear programming algorithm to obtain an optimal sparse rate distribution scheme of each layer; differentiated pruning is carried out on each layer of weight matrix according to the distribution scheme, a weight parameter with the maximum amplitude in each layer of weight matrix is reserved, the rest parameters are set to be zero, and a sparse weight matrix is obtained; according to the method, the features can be selectively pruned in the middle layer, so that fine-grained optimization of a large language model is realized.

The invention discloses a feature selection method and a data reconstruction method for shoreline image feature selection. The method comprises the following steps: S1, integrating original data into a sample data set matrix X; s2, initializing a reconstruction transformation matrix W, a transformation matrix Q, an eigenvalue regression coefficient matrix A, a coordinate basis matrix B, a coding matrix E, an auxiliary matrix F and an auxiliary matrix D as unit matrixes, initializing a weight vector p and a mean vector v of a data set as unit vectors, and initializing a weight matrix P = diag (p); and S3, updating the coding matrix E, the auxiliary matrix F and the like based on the data set matrix X, the mean vector v of the data set, the transformation matrix Q, the eigenvalue regression coefficient matrix A, the coordinate basis matrix B and the weight matrix P. According to the invention, the accuracy of data reconstruction and the validity of feature selection can be improved.

The application provides an explainable sequence recommendation method, system and device fusing concepts and behaviors, relates to the technical field of recommendation systems, and the method comprises the following steps: obtaining a concept library and a historical behavior sequence of a user; obtaining a historical concept sequence and a concept embedding matrix according to the historical behavior sequence and the concept library; obtaining a concept weight matrix and a concept behavior sequence set according to the historical behavior sequence, the historical concept sequence and the concept embedding matrix in combination with an association strategy of concepts and behaviors; and performing sequence recommendation on the user according to the historical behavior sequence, the concept embedding matrix, the concept weight matrix and the concept behavior sequence set to obtain an explainable sequence recommendation result. By introducing the association strategy of concepts and behaviors, the application not only effectively fuses abstract concept information and specific behavior information, but also dynamically adjusts the understanding and explanation of user preferences through the concept embedding matrix and the concept weight matrix, thereby improving the accuracy and explainability of the sequence recommendation result.

The invention discloses an AI data processing method and system based on edge computing, and relates to the field of edge AI processing, and the method comprises the steps: defining a weight matrix of an AI model according to an edge computing node address, inputting structured data into the AI model for multi-dimensional feature analysis, and extracting high-order data representation; analyzing the processing mode through an instruction decoding method to obtain a sparseness level, and performing parameter importance evaluation through a dynamic importance scoring method based on distribution characteristics represented by high-order data and an aging weight value to generate a deformable sparse mask matrix; performing dynamic pruning fusion on a weight matrix of the AI model through sparse matrix multiplication based on the deformable sparse mask matrix to generate a sparse data processing graph; according to the method, through node resource sensing and feature mapping, the AI model weight matrix is accurately matched with the edge computing node capability, and the operation efficiency and stability of the AI model on the edge computing node are improved.

The invention discloses a memory enhancement action recognition method and system on a Riemannian manifold and a storage medium, and the method comprises the steps: 1, collecting human body action data, and representing the human body action data as a third-order tensor; 2, expanding along three modes to obtain three corresponding matrixes; 3, calculating by adopting a human short-term memory mechanism to obtain a memory enhanced weight matrix; 4, decomposing the weight matrix through a principal component analysis method to obtain a base vector with a weight; 5, recombining, normalizing and mapping to a unit hyper-sphere, and reserving an angle relation; 6, learning modal weight parameters through a Monte Carlo Markov algorithm, and calculating geometric differences between points on the hypersphere; and 7, carrying out human body action classification by adopting a K-nearest neighbor classifier. The method effectively solves the problem of time information loss in a complex scene, and is suitable for various application scenes such as medical health, virtual reality, physical training and the like.

The invention relates to the technical field of computers, and discloses a real-time calibration method and system for an edge-side weather prediction model, electronic equipment and a storage medium, and the method comprises the steps: obtaining multi-dimensional weather time sequence data, inputting the data into a neural network model, extracting spatial-temporal features, and outputting an initial prediction value; decomposing a weight matrix in the neural network model into an amplitude component and a directional component, performing low-rank matrix updating on the directional component to obtain an updated directional component, calculating a calibration weight according to the amplitude component and the updated directional component, performing deviation correction on the initial predicted value in combination with local observation data, and generating a calibrated predicted value; historical samples and current samples are dynamically managed based on prediction uncertainty indexes, an anti-forgetting training target is constructed in combination with weight regularization constraints, and low-rank matrix parameters are updated to cope with data distribution changes; according to the invention, the reliability of real-time weather prediction service can be improved.

The invention provides a weight data processing method of a neural network model, electronic equipment and a storage medium, and relates to the technical field of machine learning, and the method comprises the steps: obtaining a weight matrix of a trained neural network model; determining a scaling factor and a zero offset of the weight matrix according to the maximum weight value and the minimum weight value in the weight matrix; based on the scaling factor and the zero offset, performing quantization processing on the weight matrix to obtain a quantized weight matrix; and splitting the quantized weight matrix into a preset number of low-rank matrixes. In the embodiment of the invention, the weight matrix is quantized based on the scaling factor and the zero offset, so that the high precision of the matrix quantization process can be ensured. Through quantization processing and matrix splitting of the weight matrix, dual compression is realized, the compression rate of the weight matrix can be improved, and the storage space of the weight matrix is reduced.

A simulation system and method for implementing a model based on an iterative neural network, the system comprising: a simulation vector-matrix multiplication circuit that encodes a weight matrix of the model based on the iterative neural network; and an analog non-linear circuit that encodes a non-linear function arranged in a feedback loop configured to return an output signal from the non-linear circuit as input to the vector-matrix multiplication circuit, wherein the system is configured to output a solution vector of values of the model based on the iterative neural network upon convergence of the system.

The invention belongs to the technical field of space intelligent modeling, and relates to a space intelligent world modeling method based on a global NURBS parameter domain. Consistent mapping and management are carried out on the multi-Patch geometric objects in a global NURBS parameter domain; generating and optimizing a control point matrix and a weight matrix for geometric expression based on a global NURBS parameter domain; realizing automatic smooth transition of the multi-Patch geometry in the splicing area based on a continuity keeping mechanism of curvature and normal constraint; executing NURBS curved surface subdivision operation according to the curvature change rate and the geometric error threshold value; and performing global solution on the control point matrixes of all the Patches, and outputting a space intelligent world model which is continuous and differentiable in a global range and has consistent parameters. According to the method, the continuity and controllability of geometric modeling are remarkably improved. The invention further provides a spatial intelligent world modeling system based on the global NURBS parameter domain.

The disclosure relates to a method and an apparatus for lightweighting of artificial intelligence models, and the method of lightweighting of artificial intelligence models includes identifying an outlier in an input vector of a layer, identifying at least one column corresponding to the outlier in a weight matrix, and quantizing weight values of columns which do not correspond to the outlier.

The invention discloses a pre-training language model deployment method, electronic equipment and a readable storage medium, and relates to the technical field of pre-training language models. The method comprises the following steps: determining an initial weight matrix and a Hessian matrix of a to-be-quantized initial pre-training language model; the initial weight matrix is used for representing the model information topology of the initial pre-training language model, and the Hessian matrix is used for representing the sensitive weight in the initial weight matrix; generating a sparse transformation matrix based on a sub-sampling random Hadamard transformation algorithm; performing incoherent processing on the initial weight matrix through the sparse transformation matrix to obtain a processed optimized weight matrix; based on the optimized weight matrix and the Hessian matrix, adaptive rounding operation is executed, and a quantized weight matrix is obtained; and determining a quantized target language model according to the quantized weight matrix, and sending the quantized target language model to the to-be-deployed device. The problem that the efficiency of deploying the pre-training language model is low is solved, and the technical effect of improving the efficiency of deploying the pre-training language model is achieved.

The invention provides a finite element twinning real-time analysis method and system based on a reduced-order proxy model. The method comprises the steps that an output result of a full-order model is obtained through offline simulation calculation; obtaining a high-precision model node coordinate based on an output result of the full-order model; kNN proximity search is accelerated through high-precision model node coordinates and a ball tree data structure to obtain an Euclidean distance, and a normalized weight matrix is obtained through the Euclidean distance; obtaining a physical field result of the low volume model based on the normalized weight matrix; performing POD decomposition on the physical field result of the low volume model to obtain a singular value decomposition result; constructing an agent model based on a singular value decomposition result, and optimizing an agent to obtain an optimized agent model; and generating an analysis result through the agent model. According to the method, the digital twin platform is accessed through the CFD dynamic agent model, unit-level rendering and pressure cloud picture switching are realized according to the second-level feedback speed, and smooth viewing and interaction of a user are supported.

The invention relates to the technical field of weighing, and discloses a truck scale weighing error compensation method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining training sample data, a first configuration parameter, a second configuration parameter and an initial model parameter; creating an initial error compensation model according to the initial model parameters; determining a target model parameter and a target parameter according to the first configuration parameter, a model parameter optimization algorithm and the training sample data; determining a target weight matrix according to the target parameter, the second configuration parameter, a weight matrix optimization algorithm and the training sample data; the target model parameters and the target weight matrix are set as model parameters and a weight matrix of the initial error compensation model respectively, a target error compensation model is obtained, and the target error compensation model is used for obtaining a target weighing result according to the input sensor signals. The problem that the accuracy of the weighing result of the motor truck scale is influenced by various factors, so that the weighing result has an error is solved.

The present application provides a deep learning convolution acceleration method using bit-level sparsity and a processor. Comprises: selecting the maximum sum of the exponents from all data pairs to be convolved as a maximum exponent; arranging mantissas of the original weights in a computation sequence to form a weight matrix, and uniformly aligning each row of the weight matrix to the maximum exponent and removing slack bits to obtain a reduced matrix, allowing essential bits in each column of the reduced matrix to fill the vacancies according to the computation sequence, after removing null rows in the intermediate matrix, placing zeros at vacancies of the matrix to obtain an interleaved weight matrix, sending the weight segments in each row of the interleaved weight matrix and the mantissa of the corresponding activation to an adder tree for processing summation, by shifting and adding the sum result to obtain a convolution result.