145 results about "Linear layer" patented technology

The invention relates to a model optimization method and device for a low-resource scene, electronic equipment and a medium, and the method comprises the steps: obtaining a pre-training model, the pre-training model being a Transform model, for any linear layer in the Transform model, connecting a pair of low-rank matrixes in parallel in the linear layer to form a LoRA increment path, and freezing a backbone network in the pre-training model; for any linear layer in the Transform model, a learnable gate is added on a LoRA increment path, and an improved LoRA module is formed; and obtaining training data, training the pre-training model according to the training data to obtain a target model, and performing semantic analysis on the to-be-processed text according to the target model. According to the method provided by the invention, through collaborative design of structural innovation and an optimization mechanism, the performance and stability of the efficient parameter fine tuning method in a low-resource scene are remarkably improved, and the method has a good industrial application prospect.

The invention relates to the technical field of satellite altimetry, in particular to an altimetry method based on a residual network multi-mode deep learning model, which can remarkably improve the robustness of SSH inversion under sea condition noise interference, and is characterized in that a fusion self-attention mechanism and a residual network multi-mode deep learning model ViTResNetMDL is established, and the model ViTResNetMDL is applied to improve the robustness of the SSH inversion under sea condition noise interference. A satellite-borne GNSS-R sea surface height inversion geometric physical model is established, a ViTResNetMDL model is composed of ResNet50, ViT and MHL-NN, the first part adopts a ResNet50 convolution structure to extract effective features from an original DDM, the second part adopts a ViT layer which captures global features in an effective scattering area DDM based on a self-attention mechanism in a Transform module, the third part is composed of linear layers, and the fourth part is composed of linear layers. And using the MHL-NN to retrieve the sea surface height.

The invention relates to an NDVI product generation method and device based on a diffusion model, equipment and a medium, and relates to the technical field of remote sensing data processing, and the method comprises the steps: obtaining a plurality of NDVI images corresponding to a target time period; each NDVI image is input into a pre-constructed diffusion generation model, and an NDVI product corresponding to the NDVI image is generated; wherein the diffusion generation model which is constructed in advance comprises a Stable Diffusion model and a LoRA fine tuning module; the Stable Diffusion model comprises a VAE module and a diffusion module constructed on the basis of a U-Net framework; and the LoRA fine tuning module is embedded in a Query linear layer, a Key linear layer and a Value linear layer in the attention structure. The method has the technical effect of effectively generating the high-time-resolution NDVI product by using the diffusion model.

The invention discloses a lane line detection method based on SDAMamba and FFCDConv, and relates to the field of lane line detection. The method comprises the following steps: preprocessing a training set lane line image to obtain an updated training set; improving the DAMamba model to obtain an SDAMamba model for extracting multi-scale features of the updated training set image; extracting depth multi-scale features by using a feature pyramid neck network; mapping the depth multi-scale feature to a Hough parameter space by using DHT and selecting a Hough feature corresponding to a lane; jointly inputting the Huffcharacteristics and the depth multi-scale characteristics into an FFCDConv module, predicting the characteristics of different lane instances, and obtaining a predicted lane position map through a lane decoder; and inputting the predicted lane position map into a linear layer to determine a final lane position, and obtaining a lane line detection image. According to the method, the problem of breakage easily occurring in a lane line detection result can be repaired in a targeted manner, and the confusion rate of adjacent lanes is remarkably reduced.

The invention provides a Bayesian nerve radiation field modeling method and system based on uncertainty perception and dynamic importance sampling. The method comprises the following steps: replacing a full connection layer in a multi-layer perceptron with a Bayesian linear layer to obtain a Bayesian neural radiation field BN-NeRF model; acquiring a data set containing house source photos of different viewing angles and corresponding camera positions, and training a BN-NeRF model by using the data set; performing preliminary coarse sampling on each light passing through the house source scene to obtain a coarse sampling point set, and performing uncertainty evaluation on each sampling point in the coarse sampling point set by adopting a trained BN-NeRF model; according to the uncertainty evaluation result corresponding to the preliminary coarse sampling, performing secondary sampling on each light passing through the housing resource scene to obtain a fine sampling point set; and integrating the coarse sampling point set and the fine sampling point set to generate a final sampling point set, calculating the color and volume density of each sampling point in the final sampling point set by adopting a trained BN-NeRF model so as to carry out volume rendering, and generating a final house viewing picture.

The invention discloses an elastic Bragg breakwater structure response prediction method, and belongs to the technical field of ocean engineering structure dynamic response prediction, and the method comprises the steps: obtaining the front and rear wave surface elevation and motion response data of a breakwater; a double-flow attention mechanism based on physical prior guidance is constructed, and wave-structure interaction is decoupled into two parallel attention flows of structural dynamic evolution and wave-structure coupling feedback; a self-attention mechanism is combined with a time decay physical prior coding structure motion feature, and a bidirectional cross attention mechanism based on a resonance characteristic is combined with a wave-structure phase relation physical prior coding coupling feature; the double-flow features are adaptively integrated through a global context fusion module, and a multi-step prediction sequence is output through linear layer decoding; and a time-frequency domain joint loss function optimization model is adopted, so that the prediction precision is improved. According to the method, physical prior is explicitly embedded into an attention mechanism and is improved by more than 16% compared with an optimal baseline model, and the generalization ability is enhanced by more than 13% under the working condition that waves do not appear.

The present application belongs to the technical field of grain informatization processing, and particularly relates to a kind of imperfect grain kernel identification method and system.Visible light image and reflectance spectrum data of grain kernel are input into identification model;From the features extracted from the visible light image and reflectance spectrum data of grain kernel respectively by the feature extraction module, the input features are obtained;The spectral Mamba branch in the identification model sequentially performs linear layer and convolution processing on the normalized input features, and then inputs the state space model to obtain the output;The spatial Mamba branch performs dimension transposition on the normalized input features, and then sequentially performs linear layer and convolution processing, and then inputs the state space model to obtain the output;The guide branch sequentially performs linear layer processing on the normalized input features, and then inputs the ConvNeXt classifier, and then adjusts the output of the ConvNeXt classifier combined with the SiLU activation function to obtain the output;According to the fusion result of the output of the guide branch, the spectral Mamba branch, the spatial Mamba branch and the input features, the identification result is obtained.

The invention provides a numerical feature embedding method based on a scale perception radial basis function and a situation awareness method for the health degree of power equipment, and the method comprises the steps: obtaining a logarithm of to-be-coded numerical data x, and obtaining a base number D and an index L; performing RBF (Radial Basis Function) expansion on the base number D to obtain the expression of the base number D; carrying out soft sub-bucket distribution on the index L to obtain a distance from the index L to each soft sub-bucket, taking the distance as a coefficient of the soft sub-bucket, and weighting each soft sub-bucket according to the coefficient of the soft sub-bucket to obtain representation of the index L; and converting the representation of the index L into two numbers by adopting a gated linear layer network, and scaling the representation of the base number D to obtain the coded representation of the numerical data x.

The invention belongs to the technical field of data mining, and discloses an aero-engine missing data filling method based on a neural network with an exogenous variable graph. Aiming at the detection parameter data and the operation parameter data, a sliding window is used for dividing according to the time dimension to obtain an intermediate matrix; the method comprises the following steps of: performing sliding window division on high-dimensional representation of input data to construct a time sequence-attribute graph; the attention is used for calculating the feature representation of the action intermediate matrix and the action time sequence-attribute graph, and a final weight matrix of the time sequence-attribute graph is obtained through combination. Convolution operation is carried out on the time sequence-attribute graph, so that the node can dynamically aggregate influences from multiple historical moments and multi-dimensional attributes, and meanwhile, the directional effect of exogenous variables on endogenous variables is effectively fused. And splicing all the convoluted feature representations, fusing time sequence information of different time points through a pooling layer, and finally calculating by adopting two linear layers and an activation function to obtain a final filling result.

The invention discloses a direction consistency constraint-based adaptive image classification method during testing. The method comprises the following steps of: 1, constructing an image classification training data set and preprocessing the image classification training data set; 2, constructing a classification model and completing pre-training; 3, decomposing the weight of each linear layer in the pre-training model to obtain a modulus length matrix and a unit direction matrix, and introducing an orthogonal matrix to construct an effective weight structure for updating in a test stage; and 4, in a test stage, calculating the difference between the model output features and the source domain features in terms of statistics, constructing a loss function with statistics alignment as a target, and on the premise of keeping the unit direction matrix unchanged, updating the modulus length matrix and the orthogonal matrix to realize adaptive adjustment of the image classification model during test. According to the method, the problem of performance degradation caused by input distribution change is effectively relieved, and the classification precision of an image classification model under a complex disturbance condition is remarkably improved.

A deep learning model based on attention using an embedding scheme for continuous variables of tabular data. A method of constructing the deep learning model based on attention includes converting tabular data of structured data having a mixture of categorical variables and continuous variables into embedding values and training a network model including a transformer block, a linear layer block, and a sharing function for the sharing of an attention between the transformer block and the linear layer block by using the embedding values.

The application provides a matrix operation accelerator combining wavelength division multiplexing and MZI cascade network, relates to the field of optical neural networks, and comprises an input signal layer, a weight signal layer, a summation layer and a nonlinear layer.The input signal layer is used for realizing matrix operation of optical signals through a Mach-Zehnder interferometer array; the weight signal layer is used for applying an electrical signal to a micro-ring modulator array to adjust a weight signal; the summation layer is used for separating the results of the action of the optical signals of different wavelengths through the weight signal; and the nonlinear layer is used for converting the optical signals into electrical signals through a photodetector array to realize a nonlinear activation function in the electrical domain.The application introduces N different wavelengths in the network formed by MZI cascade, so that the number of times of executing matrix operation is increased by N times each time, high-speed convolution operation is facilitated, and the size of the micro-ring modulator is relatively small, so that the energy efficiency and area ratio of the MZI cascade network calculation can be effectively increased.

The application discloses a kind of visual cognitive drive small sample image classification method, system and medium, the method includes pre-training stage and meta-learning stage;The pre-training stage includes: obtaining initial training set, data enhancement is carried out to the initial training set, obtains training dataset;The training dataset is embedded coding, mapping is carried out to embedding by linear layer, pre-trains the classification task of upstream;The meta-learning stage includes: the training dataset is divided into support set and query set;Image of the support set and the query set is mapped to embedding space, obtains corresponding embedding vector representation;The similarity of embedding in high-dimensional space is evaluated by relationship network, and the classification result of small sample learning output is obtained: the application can solve the problem that embedding space cannot be correctly perceived and the relationship between support set prototype and query set embedding is not accurate in the prior art.

Tuning weights of a neural network of a model for processing input of the model representing information about a technical system and outputting an output of the model for operating a technical system. The model includes a linear layer for mapping a multidimensional input of the layer depending on the weights to a multidimensional output of the layer. The model is configured to determine the input of the layer depending on the input of the model, and to determine the output of the model depending on the output of the layer. A method includes providing training data include the input of the model and a ground truth for the output of the model corresponding to the input of the model in the training data, providing a set of tuning methods for tuning the weights, determining the principal components decomposition of a weight matrix including the weights.

The invention discloses a traffic flow long-time-sequence prediction method based on a graph convolutional network, and aims to solve the problems of poor precision and large calculation overhead and memory overhead of an existing traffic flow prediction method under long-time-sequence prediction requirements. Firstly, a training sample is constructed for preprocessed data through a sliding window strategy; secondly, a traffic flow long-time-sequence prediction model based on the graph convolution network is constructed, the model is formed by stacking a plurality of TimeModule layers, after passing through a fast Fourier transform and periodic graph convolution module, weighted merging is carried out according to amplitude, output processed by the plurality of TimeModule layers passes through a linear layer, and predicted traffic state data is output; carrying out model training by taking a mean absolute error as a loss function; and finally, inputting the traffic state data to be analyzed into the model to obtain predicted traffic state data. Through verification, in a long-time-sequence (60 time steps) traffic flow prediction task, the model has the performance of high prediction structure precision, small parameter quantity, short training time and the like, and has good generalization ability.

The invention relates to a communication traffic prediction method based on cyclic shift, and the method comprises the steps: S1, dividing a traffic sequence into unequal-length fragments according to different time periods, carrying out the standardization through downsampling, inputting the standardized fragments into a numerical mapping and position embedding module, and obtaining a representation matrix; s2, performing cyclic shift operation on the divided adjacent flow sequences by adopting a shift mode, establishing sparse space-time correlation, and performing primary self-attention calculation on the shifted flow sequences along the time dimension to obtain a space-time representation matrix; and S3, after reverse cyclic shift is performed on the space-time representation matrix obtained in the step S2, a fragment is restored to an initial position, and a final prediction result is output after the fragment passes through a flattening layer and a linear layer. Compared with the prior art, the method has the advantages that the prediction efficiency is effectively improved while the prediction accuracy is ensured, and the like.

The application relates to a collaborative knowledge graph recommendation model construction method enhanced by contrast learning. Three contrast learning tasks are designed to supplement the recommendation supervision task, so that the problem that the node representation learned by the graph neural network is inaccurate due to the problems of sparse supervision signal, long tail effect, noise interference and the like can be alleviated. Meanwhile, in order to make the contrast learning more beneficial to the recommendation task, a multilayer perceptron containing two linear layers is introduced between the node representation and the contrast learning to help the contrast learning training. The application promotes the development of the knowledge graph-based recommendation system and has practical significance.

The embodiment of the invention relates to the technical field of artificial intelligence chips, provides a quantification method and device after large model training, a medium and a program product, and aims to reduce the storage burden of a video memory and improve the quantification speed of a large model. The method comprises the following steps: determining a smoother of a target operator in the large model; and when the layer normalization operator does not exist between the target operator and the adjacent previous operator, migrating the smoother of the target operator to the weight of the linear layer of the target operator, migrating the reciprocal of the smoother of the target operator to the weight of the linear layer of the previous operator, and quantifying the target operator.

This invention discloses a method, apparatus, and medium for identifying protein-protein interaction sites. The method constructs a GHGPR-PPIS model, which includes five sequentially connected GraphHeat-GPR modules, a linear layer, a fully connected layer, and a softmax layer. Built upon a graph convolutional network, it employs a hot kernel and integrates generalized PageRank technology and edge self-attention feature processing blocks. This fully utilizes hidden information in the protein graph, significantly improving the performance of protein-protein interaction site prediction. Compared with other competing models, it not only reduces the number of layers and training parameters but also slightly improves testing performance, generalization ability, and practical application capability.

The application discloses a document correction method and device and a storage medium. The application obtains an image document, performs convolution compression processing on the image document through a lightweight network architecture to obtain an output image; the lightweight network architecture comprises at least one lightweight network, each lightweight network comprises an SE network and a residual network, the input of the lightweight network is subjected to first convolution processing through the residual network, the first convolution processing result is subjected to compression activation processing through the SE network, the output of the lightweight network is obtained according to the input of the lightweight network and the compression activation processing result, and the lightweight network architecture can ensure a certain accuracy rate and speed up the processing speed through the lightweight network; the output image is subjected to pooling processing, the pooling processing result is subjected to classification weighting processing according to multiple linear layers to obtain a corrected document, and the integration of the multiple linear layers is beneficial to improving the robustness and obtaining a corrected document with good effect.

The embodiment of the invention discloses a communication signal multi-parameter intelligent parallel extraction method. The method comprises the following steps: firstly, acquiring a to-be-measured signal and extracting an I / Q component; processing the I / Q components through a multi-scale feature extraction structure to obtain high-dimensional features; extracting shared features from the high-dimensional features through a plurality of residual block groups, inputting a plurality of parallel task branches, and enhancing the shared features in the branches by using a channel attention mechanism to obtain output features; the output features of all task branches are transformed through a learnable linear layer to form a key matrix and a value matrix, the current task features are transformed through a learnable linear layer to form a query matrix, and the input features are segmented into a plurality of heads by means of a multi-head attention mechanism; each head obtains output of a single head based on zoom dot product self-attention operation, and results of all the heads are integrated to obtain enhanced features of the current task; and determining a communication signal parameter prediction label according to the enhanced features, and realizing multi-parameter efficient parallel extraction.

Disclosed are systems, apparatuses, processes, and computer-readable media for model training. A device may process, using a linear layer, an embedding generated from a first output token and input features to generate first features, wherein the first output token is generated by a previous iteration of a token predictor and wherein the input features are generated by a previous iteration of a decoding layer. A device may process, using the decoding layer, the first features to generate second features having first dimensions. A device may process, using a down-projection layer, the second features to generate third features having second dimensions smaller than the first dimensions. A device may generate, using the token predictor and the third features, a second output token.

The invention relates to the technical field of reservoir yield increase and transformation, in particular to a TRA-SR architecture-based refracturing target well recommendation method, which comprises the following steps of: collecting production data of a to-be-fractured low-efficiency well, a corresponding expert score and data of a refractured well with a good production condition as a target well construction data set; through a Transform embedding layer, each well is converted into a dense initialization vector, and position information is added to identify a sequence relation; the expert score is integrated into a multi-head self-attention mechanism of a Transform encoder part, and global preference is calculated; calculating the similarity between the to-be-fractured inefficient well and the re-fractured well as local preference; the global preference and the local preference are integrated into mixed preference through a linear layer; and after the mixing preference is obtained, a comprehensive score of the to-be-fractured low-efficiency well is calculated to recommend a well suitable for repeated fracturing. According to the method, the target well suitable for refracturing can be effectively evaluated and screened, and the method is beneficial to promoting treatment of low-efficiency wells, restoring the productivity and improving the recovery efficiency.

The invention provides a malicious user identification method and system giving consideration to privacy protection in a social network, and relates to the technical field of network privacy security, and the method comprises the steps: carrying out the structure perception sub-graph segmentation of a social network graph through an METIS algorithm, dividing an original graph into a plurality of sub-graphs, minimizing the number of edges crossing the sub-graphs, and keeping the scale balance of the sub-graphs; constructing a privacy perception GNN of an integrated gating residual attention module, wherein the privacy perception GNN comprises a privacy perception linear layer and a gating residual mechanism; based on differential privacy stochastic gradient descent framework training, combining an adaptive noise scheduling strategy, dynamically adjusting the noise scale according to privacy consumption deviation, and performing closed-loop control budget to obtain a trained model; and malicious users are identified through the trained model. According to the method, the problem of performance reduction caused by fixed noise injection and noise amplification is solved, and efficient and robust identification of malicious users is realized while differential privacy constraints are met.

The invention discloses a security reasoning method and system for privacy protection machine learning, and relates to the technical field of privacy protection machine learning, and the method comprises the steps: calculating a secret share according to a model input matrix and sand secret sharing and random vector compression secret sharing of model parameters, reconstructing a secret value and packaging to obtain TP secret sharing of a model linear layer operation result; the method comprises the following steps of: performing mask calculation on an input matrix of a nonlinear layer in TP secret sharing by adopting a random number to obtain a mask random number, calculating TP secret sharing at a first different position in bit representation of the mask random number and a required mask random number so as to determine the highest bit of the input matrix of the nonlinear layer, calculating an RELU function, and calculating a truncation result according to the RELU function, and a nonlinear layer output result is obtained based on a truncation result and a random mask in TP secret sharing, so that the communication and calculation overhead of a security reasoning online stage is reduced.

The present disclosure discloses a method, an apparatus and a computer program product for a neural network. The method comprises: during neural network training, receiving an input tensor at a linear layer in the neural network; and performing mixed precision quantization on the input tensor to obtain a quantized tensor, wherein the quantized tensor has data of different precision types. An apparatus, a computer program product and a non-volatile computer readable storage medium corresponding to the method are provided.

The invention provides a low-complexity human body motion reconstruction method based on a sparse inertial measurement unit, and belongs to the technical field of virtual reality and three-dimensional human body motion reconstruction, and the method comprises the following steps: obtaining a data set required for reconstruction training based on human body postures, carrying out time modeling through a time sequence encoder, updating joint features on a human body skeleton graph, and obtaining a reconstruction result; a skeleton is divided into a trunk and four limbs according to a human anatomical structure, global rotation of a root joint and local rotation of each joint are respectively predicted by a partition kinematics regression head, low-rank decomposition is introduced into a large-scale linear layer to compress model parameters, and forward kinematics is utilized to recover three-dimensional joint positions of the whole body. In the training process, a two-stage teacher-student distillation model framework is adopted, a teacher network is trained through real labels, and then joint rotation and joint positions output by a teacher are used as soft targets to jointly restrain a student network through rotary distillation and position distillation. While the parameter quantity is reduced, the reconstruction precision and the motion smoothness of the whole body are improved.

The invention discloses a sound signal periodic feature extraction method, a network model training method, a storage medium and equipment, and belongs to the technical field of sound event detection. The objective of the invention is to solve the problems of high sensing difficulty and poor decoupling effect of overlapped acoustic events in the current acoustic detection process. The method comprises the following steps: for a sound signal i, mapping the sound signal i to a low-dimensional space through two different linear layers to obtain p and g, and respectively carrying out expansion convolution operation on p and g to obtain pconv and gconv; for p and g, feature coding is carried out based on a Fourier basis function and a gating mechanism to obtain Fourier features, for pconv and gconv, Fourier features are obtained in the same mode, and Hadamard product is carried out on the pconv and the gconv to obtain representation of periodic features. And in the training process of the corresponding model, performing reconstruction error on the sum and the original signal i, respectively calculating two norms of the sum, and adding the two obtained two norms to obtain a Fourier series regular term for training the model.

The invention discloses an interactive camouflage target segmentation method based on an adapter, which comprises the following steps: dynamically adjusting the attention of a model to different frequency domain characteristics, refining an originally repeated segmentation process into a process of progressively understanding a target segmentation process according to a sequence of'integrating first and then details', and clicking an intensifying mechanism to improve the segmentation accuracy of a target. And the effect of the user prompt in accurately understanding the segmentation target is enhanced. The method mainly comprises a feature extraction module and an interactive segmentation module. In the feature extraction module, the features of different frequencies of the image are extracted through two paths, a pre-trained ViT frame pays more attention to the high-frequency features of the image, and a branch added with adapter fine tuning training pays more attention to the low-frequency features due to the features of a convolutional layer. In the interactive segmentation module, the features extracted by the branch added with the fine tuning of the adapter are subjected to cross attention operation with click embedding, and the expression ability of the features is increased through a linear layer to obtain enhanced features. And finally, fusing the two branches to obtain a prediction mask through a decoder.

The invention relates to the field of computer vision and machine learning, in particular to an image classification transfer learning method and system based on a solidified PCA-PEDCC linear layer. According to the method, feature dimension reduction and category calibration are realized by solidifying a PCA-PEDCC linear layer, cross-domain feature calibration is combined, negative migration and small samples are effectively relieved, and the classification accuracy in a cross-domain scene is improved compared with a traditional method; secondly, a solidified linear layer does not need to be updated, fine adjustment parameters are reduced through layered migration, training time is shortened, model parameters are reduced, edge equipment and low-hardware-resource scenes can be adapted, and deployment cost is reduced; and meanwhile, a traditional linear layer design and an all-parameter fine tuning mode are broken through, PCA and PEDCC are integrated and solidified, feature extraction and classification adaptation are considered, an obvious technical difference is formed with an existing transfer learning method, and repeated design is avoided.