41 results about "Sparse methods" patented technology

The invention relates to the technical field of target detection of automatic driving, in particular to a 4D multi-target sensing and tracking method and system based on sparse representation, which is an efficient 3D target detection algorithm, and through dynamic interaction of sparse 4D query vectors and multi-view and multi-scale features and in combination with a time sequence instance denoising and depth sensing enhancement module, the accuracy of target detection is improved. And automatic driving 3D perception with high precision and low calculation amount is realized. Benefited from a sparse query architecture, the method also inherits the advantage of high efficiency of a sparse method while keeping high precision. Through a series of targeted optimization, the generalization ability and robustness of the model in various special scenes are significantly enhanced.

The invention discloses a heterogeneous data efficient federated learning method for intelligent manufacturing fault diagnosis, and the method comprises the steps: carrying out the two-stage federated learning based on a multi-agent generative adversarial network MAD-GAN frame for the fault detection of a bearing vibration signal; in the first stage, the MAD-GAN model is trained in the federated learning environment, the data sample expansion process can effectively relieve the modal collapse problem, and in the second stage, the federated MAD-GAN model is utilized to generate data samples with high quality and diversity for a client, so that federated learning model training is performed, and the generalization ability and diagnosis precision of the model are improved. In order to further reduce communication overhead, an advanced model compression technology is introduced in the second stage of the invention, and a sparse method and a quantization method such as Top-k are combined. Redundancy in the communication process can be eliminated, the data transmission quantity is reduced, the problem of non-independent identically distributed data in industrial fault diagnosis can be solved, and the communication efficiency is remarkably improved.

The invention provides a high-resolution DOA (direction of arrival) estimation method and system, and the method comprises the steps: updating a model covariance matrix based on a to-be-corrected space power spectrum and noise information of the iteration of this round, thereby obtaining a direction dependence weight factor, and carrying out the estimation of the direction dependence weight factor. And according to the direction dependence weight factor of the current round of iteration, the to-be-corrected spatial power spectrum, the model covariance matrix and the sample covariance matrix, predicting the to-be-corrected spatial power spectrum and noise information of the next round of iteration, and finally obtaining the spatial power spectrum. According to the method, on the premise that the number of prior information sources is not needed, the problem of spectrum peak merging of a traditional sparse method under the condition of adjacent information sources is solved, stable separation of near-angle incoming waves is achieved, and the high-resolution performance of DOA estimation is remarkably improved.

The invention discloses a pulse neural network sparsification method based on a mask mechanism and adaptive threshold adjustment, and belongs to the technical field of neuromorphic calculation and deep learning. In order to solve the problems that the number of redundant pulses is large and energy efficiency is limited in an existing pulse neural network, the invention provides a method of combining a biologically inspired block mask mechanism and a self-adaptive threshold adjustment strategy so as to realize energy efficiency balance of the neural network. The method comprises the following steps: firstly, carrying out space localization mask processing on a pulse sequence by adopting a block mask mechanism designed based on a biological quantum synaptic failure theory, and reducing redundant pulses; and then, dynamically adjusting a neuron distribution threshold value through a lightweight strategy network to realize dynamic heterogeneity adjustment of different neurons. Finally, through the synergistic effect of the block mask and the adaptive threshold strategy, the network performance is maintained and even improved while the number of pulses is effectively reduced and the energy consumption is reduced. According to the method provided by the invention, the calculation energy consumption and redundancy overhead of the spiking neural network can be remarkably reduced, and the method is suitable for low-power-consumption intelligent sensing tasks on resource-constrained equipment.

The invention discloses an X: M dynamic attention sparseness method and device, and the method comprises the steps: carrying out the prediction of an attention score matrix, and carrying out the X: M structured pruning based on a prediction result, so as to obtain a pruning result; and updating the minimum line number and the index line number in the cache based on a pruning result, adjusting a scheduling threshold, and storing the index block number meeting the condition into a priority queue so as to map the corresponding index to a PEA processing unit array for execution. According to the method, the storage and calculation overhead of attention prediction is greatly reduced, and meanwhile, X: M is combined with a scheduling algorithm, so that a processing unit in an application-specific integrated circuit can keep an extremely high utilization rate under different sparseness.

The invention discloses a two-stage sparse method based on a small sample rapid sparse large-scale language model, and the method comprises the following steps: S1, mask initialization: based on a plurality of importance indexes for measuring parameters, employing a plurality of mask initialization methods to initialize a mask of a language model; s2, reordering the masks: reordering the masks based on the small sample calibration set and the straight-through estimator; s3, mask adjustment: based on the small sample calibration set and the reordered mask matrix in the step S2, taking an original model as a teacher model, taking a sparse model as a student model, realizing a block-level self-distillation process of the large language model in a time dimension before and after sparsification, and obtaining the sparse model; the method consumes less computing power, uses a smaller data set, occupies less GPU memory while realizing considerable performance, does not influence the reasoning speed of the sparse model, and has remarkable practical value and application prospect.

The invention relates to a data processing technology, in particular to an accelerated reasoning method of a large language model, computing equipment for implementing the method, a non-instantaneous computer readable storage medium and a computer program product. According to one aspect of the invention, an accelerated reasoning method of a large language model is provided, the large language model comprises a plurality of sub-layers connected in a cascade mode, the sub-layers are multi-head attention layers or feedforward neural network layers, and the method comprises the following steps: A, determining a corresponding sparsity for each sub-layer by using a dynamic sparse optimizer; b, the reasoning request is processed through sparse sub-layers, and the sparse mode is that for each sub-layer, a dynamic sparse predictor associated with the sub-layer is used for predicating the reasoning request through the dynamic sparse predictor associated with the sub-layer; an attention head or neuron that needs to be activated is determined based on the sparseness of the sub-layer and the context to a previous sub-layer input to the sub-layer and the determined attention head or neuron is activated.

This invention discloses a sparse method and related equipment for quadrant amplitude mapping arrays. The method, through recursive partitioning and a deterministic amplitude mapping mechanism, eliminates the randomness of traditional probabilistic methods, ensuring the uniqueness and stability of the output layout under the same input conditions, thereby significantly improving the repeatability and engineering reliability of the design results. The method in this embodiment achieves high-precision matching between amplitude distribution and cell position through sparse cell allocation based on recursive quadrant partitioning, significantly reducing sidelobe levels while lowering the standard deviation of performance fluctuations. Furthermore, the method in this embodiment is independent of specific array geometry and naturally supports rectangular, circular, and irregular arrays with internal obstacles through its recursive processing mechanism, demonstrating excellent versatility and flexibility, and can be widely applied in the field of data processing technology.

The present invention discloses a neural network retraining and gradient sparsification method based on memristor aging perception, and relates to a design for extending the working life of memristors and improving the accuracy of cross array reasoning. For the problem of reduced accuracy of hardware online reasoning after cross array aging, by using known memristor aging information, the maximum value of programmable weights under the current aging condition is calculated, and the neural network model is retrained accordingly, thereby improving the accuracy of current hardware online reasoning; during the retraining process, the network weights exceeding the maximum value of programmable weights are automatically truncated; for the problem of limited memristor life, the sparsity of neural network derivatives is used, and derivatives with small absolute values ​​are discarded during the hardware adjustment process, thereby ensuring that the memristor corresponding to the small derivative is not applied with voltage, preventing the aging process of the memristor and extending its service life.

The invention discloses a battery simulator circuit health assessment method based on star computation-A3C, and the method comprises the steps: carrying out the normalization processing of a high-frequency current signal, and carrying out the signal reconstruction through a period sparse method, and obtaining period sparse data; a recurrence plot method is adopted to convert the feature map into an evaluation feature map; an improved star operation convolutional network is introduced, a multi-thread Actor-Critic cooperation mechanism of an A3C algorithm is combined, an Actor network adaptively adjusts a feature fusion weight, a Critic network evaluates a feature fusion value, and high-order feature association is dynamically captured; and a support vector machine optimized based on an energy valley algorithm is adopted to realize battery simulator circuit health state evaluation. According to the method, the health state of the battery simulator circuit is evaluated through feature collaborative optimization and parallel computing acceleration of the star computation-A3C, and the method has remarkable technical advantages and application value.

Embodiments herein describe a sparse-dense-sparse (SDS) process that achieves a better pruning scheme that benefits from pruning-friendliness relative to one-shot pruning schemes. The SDS process performs a first pruning to generate a sparse ML model followed by reconstruction to generate a re-dense ML model, followed by a second pruning to generate another sparse ML model. By pruning a ML model and then re-constructing the ML model, the ML model can be made more pruning-friendly by performing data and / or weight regularization. As a result, performing the second pruning in the SDS process can result in a smoother weight distribution and lower perplexity relative to one-shot pruning.

The application discloses a speed sampling sparse method and a path planning algorithm thereof, a storage medium and an electronic device, and comprises the following steps: obtaining a speed sampling interval through a current speed of a robot, a maximum limited speed, a minimum limited speed, a control period, a maximum acceleration and a minimum acceleration; obtaining a probability density sampling interval through the speed sampling interval and the current speed of the robot; obtaining a distribution interval corresponding to the probability density sampling interval through a cumulative distribution function of a Gaussian distribution; equally dividing the distribution interval to obtain a sampling point in the distribution interval; obtaining a sampling point speed corresponding to the sampling point through an inverse function of the cumulative distribution function of the Gaussian distribution; obtaining a sampling speed of the robot through the sampling point speed and the current speed; and the above technical scheme reduces the calculation power consumption of a path planning algorithm caused by equidensity sampling speed, controls the accuracy of the sampling speed, and balances the calculation power consumption and the control accuracy.

The invention discloses a fine-grained distributed training method and system based on gradient quantization sparse compression, proposes a dynamic tensor fusion technology, automatically searches an optimal tensor fusion buffer threshold by using a binary search method, merges gradient tensors of a plurality of layers into a larger buffer for transmission, does not need manual adjustment and optimization, and improves the training efficiency. And the optimal balance between the communication delay and the bandwidth utilization rate is realized. A fine-grained pipeline scheduling mechanism for communication decoupling is provided, a gradient synchronization process is decoupled into a sparse communication task in a back propagation stage and a quantitative communication task in a forward propagation stage, fine-grained overlapping with forward and backward calculation tasks is realized, the utilization rate of calculation resources is maximized, and communication delay is completely masked. In the back propagation stage, a fragmentation Top-k sparsification and AlltoAll routing strategy is adopted, transmission of a large number of zero values is avoided, and the communication complexity of back propagation is reduced; in the forward propagation stage, a low-bit sparse quantization and AllGather strategy is adopted, and the bandwidth bottleneck problem caused by the fact that a traditional sparse method becomes dense back in the parameter synchronization stage is solved.

The invention relates to the field of neural networks and federated learning, and provides a communication compression method and system combining quantization and sparse algorithms, and the method comprises the steps: dividing a model gradient into a plurality of vectors with different dimensions according to the hierarchy of a neural network; performing data compression on all vectors through a quantization algorithm; transmitting the quantized and compressed data to a parameter self-checking module, and determining whether the gradient of the trained model is uploaded or not according to a self-adaptive threshold value and the quantized gradient contribution degree, so as to achieve the aim of sparse communication; and quantization errors and unuploaded gradient models are locally accumulated at working nodes, so that the influence of communication compression on model convergence precision is reduced. The combination of gradient quantization and a communication sparse method is used for solving the problems that the existing communication compression strength is not enough, the model training precision is reduced and the like.

According to the method, sparse vectors of intelligent reflecting surfaces serve as solutions to be optimized, a quantum dune cat method is adopted for optimization, a quantum coding technology is introduced on the basis of the dune cat method for discretization processing, and the quantum position of each dune cat serves as a sparse vector solution of a set of intelligent reflecting surfaces; for a sparse vector solution of each group of intelligent reflecting surfaces, simulating a signal transmission process through a virtual signal transmission system, taking an information transmission rate of a receiving end as a fitness function, introducing a penalty function, and taking a condition that the information transmission rate of an eavesdropping end is smaller than the maximum information transmission rate allowed to be received by the eavesdropping end as a constraint condition; if the constraint condition is not met, correcting the fitness function value by using a penalty function; and after iteration is finished, outputting the quantum position of the global optimal dune cat as an optimal sparse vector of the intelligent reflecting surface, and realizing optimization of an intelligent reflecting surface sparse scheme.

The present invention provides a method and system for spectral graph sparsification in a 3D scene graph, the method comprising: sensor data collection; scene graph construction; feature analysis and feature extraction; construction of a Laplacian matrix of the scene graph; calculation of eigenvalues ​​and eigenvectors; spectral sparsification; and verification of the connectivity and validity of the sparse graph. The present invention solves the technical problems of node redundancy in large-scale three-dimensional scene graphs, which leads to a significant increase in the scale of calculation and storage, and high complexity of the scene graph.

The invention provides a model sparse method and device, electronic equipment and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: adding auxiliary parameters into an original model by the electronic equipment to obtain an intermediate model; the auxiliary parameters are used for adjusting the precision of the intermediate model. And the electronic equipment adjusts the auxiliary parameters in the sparse process of the intermediate model to obtain a target model. The original model, the intermediate model and the target model can be collectively referred to as neural network models. According to the embodiment of the invention, in the process of sparse processing of the neural network model, the precision of the neural network model can be improved by adjusting the auxiliary parameters, and the loss of model performance caused by sparse processing of the model is reduced; and the auxiliary parameters are adjusted, and all parameters of the sparse neural network model do not need to be adjusted, so that the number of the parameters needing to be adjusted is reduced, computing resources consumed in the model rarefaction process can be reduced, the model rarefaction speed can be increased, and the efficiency of the model rarefaction process can be improved.

The application discloses a kind of support dynamic sparse and static sparse hardware accelerator, comprising: data segmentation and reordering module is used to the input data is segmented to adapt to the size of spatial accelerator hardware;Spatial accelerator module is used to support the operation of sparse attention mechanism;Weight summation module is used to support the segmentation of hybrid attention mode, the result obtained by the calculation of segmented input data is merged to obtain the final output;Pattern matching module includes matrix multiplication operation module, double tuning sequencer and sliding window comparator, for hybrid attention mode matching.The application simultaneously supports static sparse method and dynamic sparse method, and can carry out efficient static and dynamic sparse attention calculation.Saves at least 6.1 times of calculation amount, while maintaining the accuracy of output result.

This invention belongs to the field of radar signal processing and intelligent sensing technology, specifically relating to a target protection mask prior-guided low-rank sparse reconstruction method for FMCW radar interference suppression. The method includes the following steps: 1. Acquiring FMCW radar observation signals and preprocessing them; 2. Performing time-frequency transformation on the preprocessed FMCW radar observation signals to extract basic features; 3. Accurately locating the interference region in the time-frequency domain, forming a binary mask prior, and constructing a mandatory target protection region; 4. Constructing a low-rank sparse matrix and a target protection low-rank sparse decomposition model based on mask prior guidance; 5. Solving the target protection low-rank sparse decomposition model to obtain the decomposition results; 6. Reconstructing and post-processing the decomposition results, outputting the interference suppression results, and evaluating the effect. This method combines the intuitiveness of time-frequency detection methods with the reconstruction capability of low-rank sparse methods, reducing the probability of target echoes being misclassified into sparse terms.

The present invention discloses a two-stage sparsification method for rapidly sparsifying a large language model based on a small sample, comprising the following steps: S1, mask initialization: initializing the mask of the language model using a plurality of mask initialization methods based on a plurality of importance indices for measuring parameters; S2, mask reordering: reordering the masks based on a small sample calibration set and a straight-through estimator; S3, mask adjustment: based on the small sample calibration set and the reordered mask matrix of step S2, using the original model as a teacher model and the sparse model as a student model, implementing a block-level self-distillation process of the large language model before and after sparsification in the time dimension to obtain a sparse model; while achieving comparable performance, this method consumes less computing power, uses a smaller data set, occupies less GPU memory, and does not affect the inference speed of the sparse model, and has significant practical value and application prospects.

The application discloses a feedback information auxiliary Bayesian DOA estimation method and device based on a weighted atomic norm, and the method comprises the following steps: acquiring output information of a target tracker of a fusion center, and calculating a predicted direction of arrival (DOA) of each target according to the output information; constructing a prior interval of a DOA estimation value of each target according to the predicted DOA; and calculating the DOA estimation value of each target by using the prior interval and an atomic norm algorithm. The application converts the prior knowledge, i.e. the prior interval of the DOA estimation value of each target, into a semi-positive constraint, and then uses a meshless sparse method of atomic norm minimization to perform DOA estimation. The application combines array observation data and prior information, can obtain a high-resolution DOA estimation value, and reduces the calculation cost.

This invention discloses a sparse intelligent reflector method for a quantum deer hunting mechanism. First, a signal transmission system including a base station, intelligent reflector, and receiver is modeled to obtain the signal received by the receiver. Then, a system capacity calculation method is modeled for the sparse intelligent reflector to obtain the information transmission rate of the receiver after sparsification. The quantum velocity and position of each deer in the hunting population are initialized, and it is determined whether to perform an exploration or exploitation behavior. The updated position of each deer is recorded, and the fitness value is calculated using a fitness function to update the global and local optimal positions. Finally, the global optimal position is obtained. This invention automatically optimizes the reflector unit layout using a swarm intelligence optimization algorithm to find the optimal solution for the reflector unit layout. Furthermore, based on the original algorithm, quantum encoding is combined to discretize the algorithm, proposing a quantum deer hunting method. This invention possesses better global search capabilities and higher convergence accuracy.

The application discloses a multi-baseline INSAR phase unwrapping method, system, device and medium, wherein the method comprises the following steps: constructing a to-be-clustered data set: acquiring intercept information corresponding to each pixel according to interferograms corresponding to different vertical baselines, then combining position information of each pixel as multi-dimensional clustering features of the pixel, and taking the pixel as a to-be-clustered target to obtain the to-be-clustered data set; wavelet clustering processing: performing clustering processing on the to-be-clustered data set based on wavelet clustering; cluster result correction: correcting cluster labels of pixels in each noise cluster; cluster-by-cluster phase unwrapping: based on the corrected cluster distribution, calculating a blur vector of each cluster by using a closed solution formula or a sparse-TSPA method, and calculating absolute phases of each pixel in a corresponding cluster based on the blur vector of each cluster. When large-size interferograms are processed, the application has more advantages than existing multi-baseline phase unwrapping methods in terms of efficiency, accuracy and adaptability of a baseline ratio.

This invention provides a UPA near-field channel estimation method based on two-dimensional block sparsity, comprising: representing the UPA near-field channel matrix as the sum of the outer products of the horizontal and vertical ULA near-field steering vectors; constructing improved DFT dictionaries in the horizontal and vertical directions respectively; representing the channel matrix as a total coefficient matrix under these dictionaries, which presents a two-dimensional block sparse structure determined by the outer product of the horizontal and vertical block sparse representation vectors, thereby transforming channel estimation into a two-dimensional block sparse signal recovery problem; and solving the problem using the 2D-PCSBL algorithm, in which the accuracy parameter of each sparse coefficient in the prior distribution is determined by the weighted sum of its own hyperparameter and the hyperparameters of its two-dimensional neighbors, to capture the sparse coupling characteristics of the UPA near-field channel in the horizontal and vertical dimensions. This invention achieves high-precision channel estimation with fewer pilots and a lower signal-to-noise ratio, and its computational complexity is comparable to that of the one-dimensional block sparse method.

The invention discloses a B-Scan atlas optimization data sparse method in a ground penetrating radar time mode and a related device, and relates to the technical field of ground penetrating radar signal processing, and the method comprises the steps: determining the instantaneous dragging speed of a ground penetrating radar during collection based on a front A-Scan data association model and a rear A-Scan data association model; establishing a nominal speed judgment model according to the collection frequency and the number of sampling points; smooth filtering is carried out on the instantaneous dragging speed set, the average speed and the standard deviation are calculated, and the speed confidence coefficient is calculated; calculating a comprehensive difference value of the adjacent data frames, and removing one frame in the adjacent data frames based on a self-adaptive redundancy judgment threshold function; calculating a sparse coefficient based on the nominal speed judgment model and the instantaneous dragging speed set; and if the sparse coefficient threshold value is greater than or equal to the sparse coefficient threshold value, performing sparseness and data optimization on the data frame to obtain an optimized A-Scan data sequence, and generating a B-Scan atlas with a high signal-to-noise ratio.

The application discloses a kind of sparsification method and accelerator design for target detection network, for YOLOv2-tiny target detection network, under the condition of guaranteeing certain accuracy, the network is sparsified, and it has hardware friendliness, the designed accelerator can efficiently support the network model operation after sparsification, the designed computing unit has two kinds of calculation modes, can make full use of the network sparsity brought by this sparsification method.The core of the sparsification method is to remove the values with small absolute values in the convolution kernel unit, and to encode the positions of the remaining non-zero values, providing the computing unit with feature map data and weight matching during calculation, reducing a large amount of redundant calculation, saving storage space, improving running speed, so that the target detection algorithm can run efficiently on the resource-limited FPGA platform.

The application provides a robust sparse method for suppressing abnormal noise of seismic data, and belongs to the technical field of seismic exploration data processing. The two-dimensional seismic data is modeled as a combination of effective signals, random noise and abnormal noise, the abnormal noise is modeled as a Laplace scale mixed distribution, the EM algorithm is used to solve the model, the SPGL1 algorithm is used to obtain sparse representation in the discrete cosine transform domain, and the noise component is suppressed in the iteration updating process.

The invention discloses an Internet of Things data reconstruction method, application and storage method based on structure enhancement and learnable transformation tensor low-rank modeling, and belongs to the field of Internet of Things data processing. The method comprises the following steps: dispersing a monitoring area into grid points, deploying a sensor, and constructing data of continuous T time slots into a third-order original tensor; constructing a Hankel block matrix by sliding a window of the original tensor in a space-time direction, and stacking the Hankel block matrix to form a structure enhanced tensor; executing tensor singular value decomposition by using unitary transformation, and minimizing a transformation tensor nuclear norm to establish a low-rank constraint; constructing a low-rank tensor completion model, and expanding the low-rank tensor completion model into an end-to-end trainable network SLRTC-T2Net through an alternating direction multiplier method; and finally, reestablishing missing data by using the trained network. According to the method, the space-time coupling characteristic is fully mined through a double Hankel structure enhancement mechanism, effective coupling of structured modeling and low-rank completion is realized in combination with learnable unitary transformation, the missing data reconstruction precision is effectively improved, and the problem of base mismatching of a traditional sparse method is relieved.

The invention relates to the technical field of data gradient sparseness, in particular to a transmission data gradient sparseness method based on distributed reasoning, which comprises the following steps: confirming an original neural network model of a distributed computational node, performing gradient iterative computation on the original neural network model to obtain a plurality of model update gradient value sets, and performing gradient value aggregation on the plurality of model updating gradient value sets based on the to-be-trained model to obtain a local updating gradient value set, performing data packaging on the local updating gradient value set to obtain computational node gradient data, and updating the to-be-trained model in the reasoning main node by using the computational node gradient data to obtain a global updating model. According to the method, the training efficiency and the resource utilization rate of the global model in federated learning can be improved, and the communication efficiency between the main node and the computing node is improved.

The invention discloses a clustering algorithm-based phased array surface temperature measurement point rapid rarefaction method. The method comprises the following steps of S1, solving temperature fields of a phased array surface in a heating process and a cooling process; s2, clustering division is carried out on the temperature measurement points through a clustering algorithm, and similar temperature measurement points are divided into the same area; s3, solving the optimal temperature measurement point sparsification temperature measurement point arrangement condition and reconstruction precision under the condition of different clustering center numbers according to the clustering division condition; and S4, solving an optimal number of clustering centers through a multi-objective optimization method, and taking a temperature measurement point arrangement condition corresponding to the optimal number of clustering centers as a clustering algorithm sparsification result. According to the method, the number of the temperature measurement points of the active phased array surface is rapidly thinned, the positions of the temperature measurement points are optimized, the purpose of reconstructing the array surface temperature field with high precision by using fewer temperature measurement points is achieved, and the method has the advantages of being high in thinning speed and high in reconstruction precision.