Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

148 results about "Sparse model" patented technology

Sparse modeling is a rapidly developing area at the intersection of statistical learning and signal processing, motivated by the age-old statistical problem of selecting a small number of predictive variables in high-dimensional datasets.

Submerged arc furnace intelligent inspection system based on cloud side end cooperation

A submerged arc furnace intelligent inspection system based on cloud side-end cooperation belongs to the technical field of submerged arc furnace intelligent robot inspection, and comprises an end side device, an edge side platform, a cloud platform and a cross-layer cooperative processing module, the end side device is in communication connection with the edge side platform, and the edge side platform is in communication connection with the cloud platform; and the cross-layer cooperative processing module is used for intelligently deploying global resources of the end-side equipment, the edge-side platform and the cloud platform, predicting a future load trend by adopting a dynamic sparse LSTM model algorithm, inputting a prediction result into a security constraint PPO reinforcement learning algorithm to adaptively adjust a task and a computing power allocation proportion, and carrying out dynamic resource allocation according to an output result. According to the invention, the cross-layer cooperative processing module interacts and communicates with the cloud side, the edge side and the end side, so that a dynamic allocation strategy of resources is realized, and the problems of low resource utilization rate and poor real-time performance caused by a rigid resource allocation strategy during routing inspection are avoided.
Owner:HARBIN BOSHI AUTOMATION CO LTD

Large model lightweight reasoning deployment method under limited hardware resources

The invention provides a large model lightweight reasoning deployment method under limited hardware resources, and the method comprises the steps: quantifying the weight importance of a large model through a composite index of gradient sensitivity and activation frequency, and carrying out pruning operation in combination with an improved index weighted moving average strategy, thereby obtaining a structured sparse model; the sparse model is divided into sub-networks by adopting double rules, a routing decision network is trained, and an adaptive feature shunting architecture model is constructed; a multi-precision weight set is generated through a nested quantization technology, quantization bit width is dynamically adjusted, and edge equipment hardware parameters are adapted to complete reasoning environment initialization; after a reasoning request is received, an optimal sub-network is selected based on the trained routing decision network, corresponding weights are loaded in parallel, and a reasoning result is fused and output; and converting a reasoning result format, and dynamically optimizing a scheduling strategy based on a system real-time monitoring index. The method is compatible with a mainstream large model and a hardware platform, and an efficient and universal deployment scheme is provided for end-side AI engineering landing.
Owner:CHENGDU MINGTU TECH CO LTD

Feature fusion and refining embedding sparse Bayesian learning method and system for robot curved surface milling contour error monitoring

The invention belongs to the technical field of robots, and discloses a feature fusion and refining embedding sparse Bayesian learning method and system for robot curved surface milling contour error monitoring, and the method comprises the steps: carrying out the structural fusion of feature vectors composed of technological parameters, robot rigidity, cutting force and tracking errors into a dictionary matrix; and a sparse Bayesian learning method is combined to realize robot processing error monitoring. During model training, the posterior distribution of the weight variables can be obtained, and the expectation and variance of the posterior distribution of the weight coefficients are estimated by maximizing the posterior probability. Wherein the variance and the hyper-parameter influencing the weight sparsity are estimated by maximizing an edge likelihood function, that is, optimization is carried out by the expectation of a joint likelihood function under maximized weight posterior distribution, and the minimum value solution can be solved for the converted loss function to optimize the hyper-parameter. The posterior expectation and variance of the model weight and the hyper-parameter are iteratively updated in the training process, and the iteration process is terminated according to a set change threshold value so as to realize the sparsification of the model weight vector and the construction of the sparse model. According to the method, robot milling error monitoring of curved surface parts with different features under the variable poses is achieved, and the part measurement cost is effectively reduced.
Owner:HUAZHONG UNIV OF SCI & TECH +1

Multi-target detection method and system based on sparse reconstruction and expectation maximization clustering

PendingCN119270245ACompatible with actual application scenariosReliable multi-objective detection resultsAcoustic wave reradiationSparse modelNormal density
The invention belongs to the technical field of underwater target detection, and relates to a multi-target detection method and system based on sparse reconstruction and expectation maximization clustering, and the method comprises the steps: carrying out the preprocessing of echoes received by a linear array in a detection region, and obtaining K independent sample vectors meeting the zero-mean multivariate complex Gaussian distribution; the method comprises the following steps: carrying out uniform and equal-interval sampling on a space sector of a detection area to obtain K theta alternative angles, constructing a K theta-dimensional airspace sparse vector based on the K theta alternative angles, establishing a sparse model for judging whether a target exists in an independent sample vector, and obtaining a joint probability density function; a joint method of expectation maximization clustering based on a sparse model is adopted, and classification and estimation results of parameters in the multi-target scene are obtained through loop iteration; and obtaining an adaptive detector based on likelihood ratio test according to classification and estimation results, and realizing accurate multi-target detection. The method can automatically adapt to different target conditions, and the accuracy and reliability of detection are improved.
Owner:INST OF ACOUSTICS CHINESE ACAD OF SCI

Power adaptive digital pre-distortion method, system and device based on sparse GRU and medium

The invention discloses a power adaptive digital pre-distortion method, system and device based on sparse GRU and a medium. The method comprises the steps that input and output signals under different powers are collected, a composite feature vector is constructed through amplitude normalization and phase alignment, and a pre-distortion training target is generated in combination with indirect learning; constructing a sparse GRU neural network, introducing L1 regularization constraint, and obtaining a sparse pre-distortion model through joint optimization of mean square error and sparse constraint; partial weight updating is realized based on power grading, and weight fusion output is carried out during power interval switching so as to complete power adaptive optimization; pruning and compressing the sparse GRU model and then deploying the sparse GRU model in a pre-distortion module to compensate the nonlinearity of the power amplifier; according to the method, the model complexity is reduced through rarefaction, the dynamic working condition stability is improved through a power adaptive mechanism, pruning compression adapts to low-power-consumption hardware, high precision, low complexity and adaptive capacity are considered, and the method is suitable for the low-power-consumption and high-performance requirements of a broadband wireless communication system.
Owner:XIDIAN UNIV

Federal learning method and architecture based on dynamic sparse pulse neural network optimization

The invention relates to the crossing field of a federated learning algorithm framework, dynamic sparse training and a spiking neural network, and discloses a federated learning method and architecture for dynamic sparse spiking neural network optimization, and the method comprises the steps: employing a cloud edge end hierarchical architecture, and carrying out the training of local data on an edge device through a spiking neural network; the global model and the sparse mask received by the edge device generate a local sparse model for training; after each round of local training, the edge device uploads new model parameters to the edge server; after the edge server receives the model parameters, the model parameters are aggregated in combination with information such as historical contribution degree and the like, and are uploaded to the central cloud server; on the cloud server, the model parameters are further aggregated to generate a global model, and the global model is broadcasted to each edge server and each edge device before the next round of training. The sparse training and the spiking neural network are introduced into federated learning, so that the computing communication overhead is remarkably reduced, and the performance of the global model is improved.
Owner:ZHEJIANG GONGSHANG UNIVERSITY

Lightweight distributed model aggregation method applied to low-power-consumption chip of Internet of Things

The invention relates to the technical field of the Internet of Things, in particular to a lightweight distributed model aggregation method applied to a low-power-consumption chip of the Internet of Things, and is suitable for an Internet of Things system composed of a cloud server and a plurality of edge nodes. Each edge node learns an optimal strategy from offline data, and meanwhile, a complex teacher model is compressed into a lightweight student model through a knowledge distillation technology, so that the local calculation burden is reduced. And each edge node uploads quantized and sparse model parameters to the cloud server, and the cloud server performs federal aggregation by adopting a weighted average method to generate a global model and feeds back the global model to each terminal on the premise of ensuring data security. In order to adapt to the practical application of a low-power-consumption chip, a dynamic voltage frequency regulation technology is introduced in the hardware level, and the energy consumption and the communication overhead are effectively reduced. According to the method, the model training process of the edge nodes of the Internet of Things is optimized, and federated learning of privacy protection, efficient communication and cooperative self-adaption is realized.
Owner:XIDIAN UNIV

Multi-core vector processor-oriented sparse bert model acceleration method, apparatus and device, and medium

The invention discloses a sparse bert model acceleration method and device for a multi-core vector processor, equipment and a medium, and relates to the technical field of vector processors, and the method comprises the steps: constructing a pre-training language model for adjustment and optimization; determining the type of the graph optimization operation; the type is operator fine tuning, and the graph optimization operation of the target application scene is realized by utilizing the optimized pre-training language model and a multi-core vector processor in combination with layer normalization and matrix transposition; the type is operator fusion, operator fusion is carried out on each layer of encoders in the optimized pre-training language model based on a multi-core vector processor, and graph optimization operation on a target application scene is realized; the type is storage optimization, data storage structure optimization and data reuse design are carried out on the optimized pre-training language model based on the multi-core vector processor, graph optimization operation on a target application scene is achieved, the parallel computing capacity of the multi-core vector processor can be brought into full play, and the data reusability and the data storage effect are improved.
Owner:NAT UNIV OF DEFENSE TECH

Intelligent prediction method for intelligent jacking construction progress of bent cap

The invention relates to the technical field of intelligent construction, and discloses a bent cap intelligent jacking construction progress intelligent prediction method, which comprises the following steps: collecting multi-source real-time data such as equipment, structure, personnel and environment in a construction process, and calculating a construction entropy for measuring system uncertainty; establishing a causal state evolution model, and predicting the evolution trend of the construction entropy in a future time period; and when it is predicted that a construction progress disturbance risk exists, performing anti-fact deduction based on the model or the dynamically trimmed sparse model thereof, and generating and recommending an optimal adaptive correction strategy. According to the method, the overall uncertainty of the construction system is quantified into the construction entropy, and the prospective deduction capability of the causal state evolution model is combined, so that the conversion from the risk passive response depending on experience to the data-driven beforehand active early warning and optimal decision recommendation is realized; and improvement of scientificity, foresight and timeliness of construction risk management is facilitated.
Owner:SUZHOU TRAFFIC ENG GRP CO LTD

Sparse Transform model reasoning acceleration method based on GPU platform

The invention provides a sparse Transform model reasoning acceleration method based on a GPU platform, and the method comprises the steps: building a first storage format in a sparse Transform model, and carrying out the coding of a sparse mask of any type through the first storage format; wherein the first storage format comprises an inner layer block and an outer layer block; an outer layer block adopts a block compression sparse row structure, and row pointers and column indexes of a completely effective block and a partially effective block are recorded; the inner layer block adopts an unsigned 64-bit integer value bitmap and is used for representing masks in the block; based on the sparse mask matrix coded by the first storage format, performing sparse calculation on the query, the key and the value, and outputting a corresponding context vector; compared with a traditional sparse mask storage structure in the prior art that calculation sparsity and memory access regularity are difficult to consider when the masks present random, global or mixed distribution characteristics, the method has the advantages that support for any type of masks is achieved, storage overhead is reduced, memory access continuity is improved, good GPU parallelism is kept, and thread divergence and memory access conflicts are avoided.
Owner:CHINA UNIV OF PETROLEUM (BEIJING)

Internet of Things federal learning method and system based on dual dynamic sparse training

The invention discloses an Internet of Things federated learning method and system based on dual dynamic sparse training. The method is oriented to an Internet of Things scene, dynamic sparse training is adopted, the model is trained only by using local data, and low communication overhead and individuation can be achieved. And then, only the model parameters are transmitted to a server for aggregation, and a personalized aggregation method is provided to ensure that the personalization of the model can be kept without losing after the parameters are uploaded. By transmitting the lightweight sparse model, the problems of non-uniform equipment resources, overlarge communication overhead and the like in the scene of the Internet of Things are effectively solved, personalized aggregation is provided, the sparse models on different equipment are aggregated, and the performance is improved.
Owner:ZHEJIANG GONGSHANG UNIVERSITY +1

Shape memory alloy soft actuator control method based on data-driven modeling

The application relates to a shape memory alloy soft driver control method based on data-driven modeling, which comprises the following steps: measuring and collecting input and output data of a shape memory alloy soft driver as sample data; designing an asymmetric activation function containing a hysteresis characteristic, constructing a neural network structure and selecting corresponding parameters, combining sample data and an improved gradient algorithm for iterative training, simultaneously introducing L1 regularization to generate a sparse model, and obtaining a soft driver theoretical model; inputting current soft driver corresponding input data into the soft driver theoretical model, outputting corresponding simulation output data, and controlling the soft driver according to the simulation output data. Compared with the prior art, the application adopts a neural network model, combines an improved nonlinear activation function and a gradient descent algorithm, improves the modeling accuracy and robustness of a hysteresis system, and effectively improves the subsequent control effect.
Owner:SHANGHAI UNIV

Gene sequencing data analysis method and device, computing equipment and readable storage medium

The embodiment of the invention provides a gene sequencing data analysis method and device, computing equipment and a readable storage medium, and belongs to the field of data processing. The gene sequencing data analysis method is applied to user equipment, and comprises the following steps: encoding gene sequencing data of a user to obtain a plurality of gene sequencing plaintexts corresponding to the gene sequencing data; encrypting each gene sequencing plaintext by using the public key to obtain a plurality of gene sequencing ciphertexts; sending the gene sequencing ciphertext to a server; receiving a gene sequencing data analysis result ciphertext sent by the server; and decrypting the gene sequencing data analysis result ciphertext by using the private key, and determining a gene sequencing data analysis result of the user. And the user equipment sends the plurality of encrypted gene sequencing ciphertexts to the server, so that the privacy leakage of the user is avoided. And the server performs data analysis by using the preset sparse model, and processes the to-be-detected gene sequencing ciphertext, so that the gene sequencing data analysis efficiency is improved.
Owner:SANSURE BIOTECH INC

Two-stage sparse method based on small sample fast sparse large language model

The invention discloses a two-stage sparse method based on a small sample rapid sparse large-scale language model, and the method comprises the following steps: S1, mask initialization: based on a plurality of importance indexes for measuring parameters, employing a plurality of mask initialization methods to initialize a mask of a language model; s2, reordering the masks: reordering the masks based on the small sample calibration set and the straight-through estimator; s3, mask adjustment: based on the small sample calibration set and the reordered mask matrix in the step S2, taking an original model as a teacher model, taking a sparse model as a student model, realizing a block-level self-distillation process of the large language model in a time dimension before and after sparsification, and obtaining the sparse model; the method consumes less computing power, uses a smaller data set, occupies less GPU memory while realizing considerable performance, does not influence the reasoning speed of the sparse model, and has remarkable practical value and application prospect.
Owner:XIAMEN UNIV

Rolov5-based motor vehicle model optimization method

The invention discloses a method for optimizing a motor vehicle model based on yolov5, and the method comprises the steps: S1, preparing a data set: collecting image data containing various motor vehicles, generating a motor vehicle data set, and dividing the data set into a training set, a verification set and a test set; s2, model training: replacing a backbone network with MobileNetV3 on the basis of a YOLOv5 network structure, and inputting a data set for model training to obtain an algorithm model weight; s3, model rarefaction: carrying out rarefaction training on the model; s4, model pruning processing: pruning the sparse model, and performing weight pruning on the data; and S5, evaluating the precision of the model: carrying out performance test on the pruned model, and evaluating the precision and speed of the model. According to the method, after INT8 precision is converted by bypassing a model, the precision is quantified and improved through a calibration method, the model is optimized, the generalization ability of the model is improved, rapid edge end model deployment is achieved, and the deployment difficulty and time consumption are reduced.
Owner:NANJING HOWSO TECH

GIS latent insulation fault diagnosis method and system under strong noise

The invention belongs to the technical field of GIS fault detection, and discloses a GIS latent insulation fault diagnosis method and system under strong noise, and the method achieves the omnibearing signal capture through synchronously obtaining optical, mechanical, electrical and acoustic signals, and solves a problem of weak signal leak detection. For strong noise interference, noise and fault characteristic frequency bands are effectively separated by using multi-modal signal difference and constructing a time domain and wavelet domain cascaded double-layer dictionary. A discriminative sparse model is combined with a K-SVD algorithm to carry out end-to-end iterative optimization, sparse features with higher discriminative force are automatically mined, and the problem of deep feature mining is solved. For the problem of weak generalization ability of small samples, a classifier error term is introduced into an objective function, a joint optimization objective function fusing reconstruction and classification errors is constructed, dictionary learning and classifier training are combined into one, the generalization ability of the model is significantly enhanced, and the method is suitable for large-scale popularization and application. Therefore, timely and accurate identification of the latent insulation fault under the conditions of strong noise and small samples is realized.
Owner:XI AN JIAOTONG UNIV

Acceleration method and device based on irregular sparse model, electronic equipment and medium

The embodiment of the invention discloses an acceleration method and device based on an irregular sparse model, electronic equipment and a medium. A specific embodiment of the method comprises the steps of inputting a sparse model into a target compiler; in the layer intermediate representation, splitting a sparse weight matrix of the sparse model into sparse sub-blocks to obtain a split sparse weight matrix; based on a sparse operator performance model, generating attribute information of each sparse sub-block in the split sparse weight matrix; in the block layer intermediate representation, according to the attribute information of each sparse sub-block, performing fusion processing on each sparse sub-block; in operator layer intermediate representation, according to attribute information corresponding to each sparse block, mapping each sparse block and a calculation unit to obtain a mapping result; and generating a processing task corresponding to the sparse model according to the mapping result, and executing the processing task. According to the embodiment, end-to-end optimization is realized through multi-layer optimization, and the income boundary of the traditional irregular sparse acceleration design is broken through.
Owner:SHANGHAI QI ZHI INSTITUTE

Battery SOH and RUL joint prediction method based on HS-MH-MoE model and EIS data

The invention discloses a battery SOH and RUL joint prediction method based on an HS-MH-MoE model and EIS data. The method comprises the following steps: S1, testing and collecting battery data: collecting battery EIS data by adopting a professional impedance spectroscopy test instrument; s2, fusing the different EIS data and the corresponding SOH and RUL data; s3, extracting EIS data features by using a multi-layer multi-head sparse MLP sub-expert model; s4, generating shared feature representation; and S5, simultaneously predicting SOH and RUL of the battery. According to the method, a simple multi-layer perceptron model is used as an expert sub-model to construct an HS-MH-MoE model, and EIS fusion data is used in one model to jointly predict SOH and RUL of the battery. The method has the comprehensive advantages of multi-source data fusion, multi-layer multi-head sparse model integration, multi-target prediction, feature sharing, precision, efficiency and the like, and is a high-performance and high-efficiency prediction model.
Owner:NANTONG LE CHUANGXIN ENERGY CO LTD

Unmanned aerial vehicle model pruning method and system based on federal reinforcement learning

The invention discloses an unmanned aerial vehicle model pruning method and system based on federal reinforcement learning, and relates to the federal learning and reinforcement learning technology, and the method comprises the steps that in a pruning round, a server uses a reinforced generalization pruning method (RGP) to prune a global model, and a pruned model is obtained; the server performs screening to obtain clients participating in pre-training of the current round, and issues the pruned model or the global model to the clients of the current round; the client receives the pruned model or global model sent by the server, and performs fine tuning by using local data; and the client transmits the fine-tuned model to the server, and a sparse model aggregation method (FedSA) is used in the server for model aggregation to obtain a new global model. According to the method, the training effect of federal learning at the edge end with limited resources can be optimized, and the complexity of the training process is reduced while the federal model effect is kept.
Owner:INFORMATION SCI RES INST OF CETC

High-precision measurement method for size of ABS (Acrylonitrile Butadiene Styrene) plate mold

The invention relates to the technical field of size measurement, in particular to a high-precision measurement method for the size of an ABS plate mold, and the method comprises the steps: scanning the surface of the ABS plate mold twice, respectively obtaining all point cloud data and all sparse point cloud data, and respectively constructing a three-dimensional mold model and a three-dimensional sparse model, performing spatial calibration on the three-dimensional mold model, and dividing the three-dimensional mold model into a plurality of component models; and calculating the space deviation value, the reflection deviation value, the adjustment coefficient and the adjusted filtering radius of each part model, filtering the point cloud data on all the part models by combining a filtering algorithm to obtain a reconstructed three-dimensional mold model, and measuring the size of the ABS plate mold. By dynamically adjusting the filtering radius of the filtering algorithm, the deviation influence caused by the reflectivity difference of different materials in the ABS plate mold is effectively reduced, and the precision and accuracy of size measurement of the plate mold are improved.
Owner:ZHEJIANG JIEWEI PRECISION MASCH CO LTD

Time-varying nonlinear system rapid identification method based on beta wavelet basis function expansion

The invention provides a time-varying parameter model identification method based on beta wavelet basis function expansion. As shown in the flow in figure 1, the method comprises the following steps: firstly, establishing a nonlinear autoregression time-varying parameter model by using a non-stationary time-varying signal, and determining the order of the model by using an FPE order determination criterion; performing linear weighted expansion on the model time-varying parameters by utilizing a first-order beta wavelet basis function, and converting a model time-varying parameter identification problem into a time-invariant parameter estimation and model effective item selection problem, namely obtaining a beta wavelet expansion time-invariant parameter model; selecting effective items of the model by adopting a classical orthogonal forward regression algorithm, determining the number of the effective items of the model in combination with an APRESS cross validation standard, eliminating redundant items in the expansion model, estimating corresponding time-invariant parameters of the model, and establishing a sparse model structure; and finally, in combination with the obtained effective items and corresponding parameters, obtaining a sparse nonlinear autoregression model and time-varying parameters thereof through reverse solution. Compared with an existing time-varying parameter identification method based on main B-spline multi-wavelet basis function expansion, the time complexity of the identification process can be effectively reduced, the identification precision of a sparse model is improved, and a new thought and a new calculation framework are provided for identification analysis of nonlinear time-varying parameter signals.
Owner:BEIHANG UNIV

Model pruning method and device, electronic equipment and program product

The invention provides a model pruning method and device, electronic equipment and a program product, and the method comprises the steps: carrying out the iterative sparse processing of channels in each network layer in a pre-trained target model, and obtaining a target sparse model corresponding to the target model until the difference value between the model precision of the target model and the model precision of the sparse model corresponding to the target model in the current iteration is greater than or equal to a preset threshold value, and obtaining a sparse processing result of channels in each network layer in the target sparse model. And determining to-be-pruned channels in each network layer in the target model, and performing physical pruning on each to-be-pruned channel to remove the to-be-pruned channels in each network layer in the target model to obtain a pruned model of the target model. According to the model pruning method, the model precision of the target model after pruning can be effectively ensured on the premise of reducing the resource occupation amount of the target model after pruning.
Owner:IFLYTEK CO LTD

Sparse array flexible ultrasonic transducer design and manufacturing method

The invention discloses a method for designing and manufacturing a flexible ultrasonic transducer of a sparse array, and relates to the technical field of transducer manufacturing, a sparse model taking array element number-imaging quality-cost as a target is constructed, and a genetic algorithm is utilized to carry out non-uniform array element arrangement optimization; positive and negative electrodes are manufactured by adopting a microelectronic printing technology, an S-shaped negative electrode loop is designed to reduce crosstalk, PDMS is spin-coated in a layered manner to realize light and thin packaging, and acoustic impedance matching and bendability are improved in combination with a flexible backing material; the method comprises the following steps: simulating a wearing scene, identifying an ultrasonic signal distortion rate, calculating a signal-to-noise ratio and a resolution ratio in real time based on pulse echo data, adjusting an array element driving phase through feedback, and carrying out dynamic calibration optimization. According to the flexible ultrasonic transducer, the imaging performance and reliability are remarkably improved while the manufacturing cost is reduced through sparse, multi-objective optimization and dynamic calibration technologies, and the applicability of the flexible ultrasonic transducer in the fields of wearable medical treatment, industrial detection and the like is expanded.
Owner:JILIN UNIVERSITY

Large language model reasoning-oriented sparse reasoning method, system, equipment and product

The invention discloses a sparse reasoning method, system, equipment and product for large language model reasoning, and relates to the technical field of artificial intelligence. The method comprises the following steps: firstly, acquiring a weight matrix of a large language model, and then carrying out pruning sparsification processing on the weight matrix to obtain a sparse matrix; the storage format of the sparse matrix is converted into a bitmap coding storage format which is suitable for tensor calculation core perception and adopts a multi-level block structure to respectively correspond to different calculation granularities in a GPU / NPU hardware architecture so as to obtain a sparse model, and then the sparse model is deployed to respond to a reasoning request to perform reasoning service; sparse matrix multiplication is completed through a sparse matrix multiplication kernel based on on-chip storage bitmap coding so as to perform reasoning, so that through an innovative sparse matrix storage format and calculation optimization, the storage efficiency and the calculation performance in the reasoning process are remarkably improved, and especially in a low-sparseness scene, the reasoning efficiency is greatly improved. And the performance blank of the existing sparse reasoning framework in the field is filled.
Owner:HEBEI TSINGHUA DEV RES INST

Personalized federal learning method and system based on client clustering and sparse pruning

The invention discloses a personalized federal learning method and system based on client clustering and sparse pruning, and the method comprises the steps: firstly constructing a central server-multi-client frame, initializing a full-size cluster model, a channel mask and an incremental pruning rate, and configuring SGD training, network slimming compression and encrypted transmission; performing dynamic re-clustering in each round of training, executing channel-level pruning by the client by using cluster structure information, performing local fine tuning, and uploading a sparse model; and the server carries out channel fusion on the model, and arithmetic averaging is carried out on the shared channel weight to generate a unified cluster model to be issued. After the target pruning rate is reached, the client continues personalized training and uploads, the server only executes FedAvg in the shared channel, the private channel keeps the original weight, and heterogeneous aggregation is achieved. A lightweight model with shared knowledge and private features can be obtained through iteration, the communication and storage overhead is remarkably reduced, and the method is suitable for a privacy-friendly intelligent reasoning scene of resource-constrained edge equipment.
Owner:EAST CHINA NORMAL UNIV

Model data compression and decompression method and device

The embodiment of the invention provides a model data compression and decompression method and device. In the compression method, after a compression device obtains a to-be-compressed target model, N key view angles can be determined in M shooting view angles, and projection and back projection are carried out on position information of point clouds in the target model under the N key view angles to obtain a first sparse model; and rendering the attribute information of each point cloud in the first sparse model to obtain a second sparse model, so that the scene characterization capability of the second sparse model tends to the scene characterization capability of the target model, and then projecting the position information and the attribute information of each point cloud in the second sparse model under N key perspectives to obtain a projection result. And obtaining a first projection image set, performing image compression on each projection image in the first projection image set to obtain a second projection image set, and taking the second projection image set as compression data corresponding to the target model.
Owner:ZHEJIANG UNIV +1

Storage optimization method and device for hybrid expert model, equipment, medium and product

The invention discloses a storage optimization method and device for a hybrid expert model, equipment, a medium and a product. The method comprises the following steps: loading a target hybrid expert model into a computing chip, and obtaining various sparse model weight matrixes in the target hybrid expert model; the sparse model weight matrix comprises a plurality of compression units with set sizes, each compression unit comprises at least one effective row and at least one sparse row, and each effective row comprises at least one structured sparse calculation storage unit; generating a compression matrix set corresponding to each model weight matrix; the compression matrix set comprises a compression data matrix, an index matrix and a metadata matrix; and storing each sparse model weight matrix in the target hybrid expert model as a matched compression matrix set so as to realize storage optimization of the target hybrid expert model. According to the technical scheme provided by the embodiment of the invention, the storage resource overhead of the hybrid expert model can be effectively optimized.
Owner:SHANGHAI SUIYUAN TECH CO LTD

Rapid deployment of deep neural networks (DNNS) for edge computing via structured pruning at initialization

A method, system, and transitory computer-readable media for providing rapid deployment of Deep Neural Networks (DNNs) for Edge Computing using Structured Pruning at Initialization (SPaI). An input of a dense model and a pruning amount is received. Unstructured Pruning (UP) of the input model is performed by the pruning amount to generate a sparse model pruned by the pruning amount. The sensitivity of each layer of the sparse model to pruning is evaluated by contrasting a sparsity of the sparse model with an average sparsity of a global model to generate a structured pruning plan. The structured pruning plan is applied to the resilient layers. A remaining sensitive layers are reinitialized to produce an initialized model pruned by the pruning amount. The initialized model pruned by the pruning amount is provided as an output.
Owner:RAKUTEN MOBILE INC

A deep learning channel estimation method and device suitable for underwater acoustic sensor networks

The application relates to a deep learning channel estimation method and device suitable for an underwater acoustic sensor network. The method utilizes OFDM pilot data to establish a sparse signal model, and obtains a real-value sparse model by performing real-value transformation on the sparse signal model. An approximate message passing (AMP) estimation framework is introduced to realize preliminary recovery of channel information. Further, a model-driven deep learning framework is proposed. An ST-LAMP network based on AMP without considering sparse prior and a GGM-LAMP network considering sparse prior are respectively established. Network training is respectively performed according to a predetermined strategy. Optimal matching parameters are learned through data learning and are updated and fixed. Correct channel estimation values can be output through input of new measurement values. The problem that traditional empirical parameter setting cannot be applied to high complexity of an underwater acoustic channel, leading to deviation of channel estimation from an actual channel, is solved. The precision of channel estimation is improved. The channel estimation method has the characteristics of low complexity and strong self-adaptability.
Owner:XIAMEN UNIV

Adaptive Distillation Method for N:M Sparse Fine-tuning

An adaptive distillation method for N:M sparse fine-tuning relates to the compression and acceleration of artificial neural networks. The method includes the following steps: 1) Pruning a pre-trained model using an N:M pruning scheme based on weight size. 2) Using adaptive knowledge distillation to transfer knowledge in the pre-trained model to the N:M sparse model, wherein the adaptive knowledge distillation can automatically replace the activation values ​​with large differences in the student feature map with the activation values ​​in the teacher feature map according to the difference in activation values ​​between the student model feature map and the teacher model feature map, so as to reduce the difficulty of student model learning and accelerate the convergence speed of student model fine-tuning. The proposed adaptive knowledge distillation method can be used to accelerate the N:M sparse fine-tuning of the pre-trained model and reduce the accuracy loss caused by N:M sparsity.
Owner:XIAMEN UNIV