1159 results about "Forward propagation" patented technology

The invention discloses a quantitative training method and system for a deep neural network, and the method comprises the steps: obtaining an original neural network model and a verification data set, and generating a quantitative priority through hierarchical sensitivity evaluation and topology analysis; on the basis of the layer sensitivity matrix, differentiated quantization parameters are configured for each layer, and quantization constraints are established; constructing an error compensation mechanism, and generating an error compensation strategy; executing dual-scale adaptive quantization training, and respectively adopting an independently optimized quantization strategy for forward propagation and back propagation; dynamically adjusting quantization parameters according to the training state and the resource constraint; and evaluating model performance and resource efficiency. According to the method, through multi-dimensional sensitivity evaluation, a dynamic quantification strategy and system error compensation, the precision and efficiency of quantification training are improved, and meanwhile it is guaranteed that deployment resource constraints are met.

The invention discloses an engineering safety progress intelligent monitoring method based on multi-source data collaboration, which relates to the technical field of intelligent engineering monitoring, and comprises the following steps of: mapping multi-source engineering monitoring data into nodes and edges of a graph in real time by utilizing an incremental graph updating algorithm, generating a dynamic knowledge graph, and generating a dynamic mapping result; performing graph traversal on the dynamic knowledge graph through an association subgraph extraction algorithm, extracting a security event matrix and a project progress matrix, calculating an SPI index value by using a dynamic weighted fusion algorithm, synchronously performing multi-threshold interval grading on the SPI index value, generating an SPI early warning level, performing state coding on the SPI early warning level, and generating a comprehensive feature vector; according to the method, data of different types can be effectively integrated and a basis is provided for formulating a targeted engineering safety progress solution through an incremental graph updating algorithm and a Bayesian causal graph model, and node probability distribution characteristics are extracted by using a forward propagation layer. The method is advantaged in that the incremental graph updating algorithm and the Bayesian causal graph model are utilized to effectively integrate data of different types and provide a basis for formulating a targeted engineering safety progress solution.

The present disclosure provides a detection method for low-quality fake face images based on domain-invariant feature learning, including the following steps: S1, inputting datasets into a residual network (ResNet)-50, and forward propagating low-quality compressed input images and original images corresponding thereto; S2, obtaining high-frequency information of two datasets of different resolutions; and using the high-frequency information to compensate for features discarded in the compression process; and S3, performing T-distributed stochastic neighbor embedding (T-SNE) operation on outputs from the ResNet-50 at a final stage to obtain feature distributions of a high-quality dataset and a low-quality dataset. Domain invariance is used for inscribing correct artificial artifact traces, alleviating the overfitting on low-quality compressed images.

The invention discloses a lithium battery health state prediction method and system. The method comprises the following steps: acquiring lithium battery operation data; inputting the lithium battery operation data into a lithium battery health state prediction model for health state prediction to obtain a prediction result; wherein the lithium battery health state prediction model is obtained by collecting historical battery operation data as a sample set to train a basic model, a forward propagation algorithm, a back propagation algorithm and hyper-parameter tuning are adopted in the training process, and the basic model is a model formed by connecting an encoder of a Transform layer, a multi-layer KAN network as a decoder and an unscented Kalman filter; and outputting the prediction result. By implementing the method provided by the invention, the accuracy and stability of SoH prediction can be improved.

The invention relates to a training method, a data processing method, electronic equipment and a computer readable storage medium, and relates to the technical field of computers. The training method comprises the steps that a training sample is input into a to-be-trained model, forward propagation is executed, output of the to-be-trained model is obtained, the to-be-trained model comprises a plurality of expert layers, and a specific activation value is not stored in the forward propagation process; determining the value of a loss function according to the output of the to-be-trained model; according to the value of the loss function and the loss function, back propagation is executed, in the back propagation process of each expert layer, a recalculation task of a specific activation value and a first communication task are executed in an overlapping mode, and for each expert layer, the first communication task comprises a communication task of data interaction among multiple devices corresponding to the expert layer.

The invention provides a deep learning large model training method and system for heterogeneous equipment, belongs to the technical field of model training, and aims to solve the problem that a heterogeneous GPU cluster cannot be effectively utilized when a traditional scheme is used for training a deep learning large model. Different network layers of a deep learning large model to be trained are divided into a plurality of stages, forward propagation and back propagation calculation of all network layers in each stage is executed by independent virtual equipment, and meanwhile, utilization of GPU resources of different structures is coordinated in combination with a proposed hybrid parallel training strategy, so that efficient model training is realized.

The invention relates to a resource-limited large model heterogeneous training method, computer equipment and a storage medium, and the method comprises the steps: determining a first activation amount according to a dependency relationship between calculated amounts corresponding to the activation amounts and / or the calculated amount and the storage amount of each activation amount in the activation amounts generated when an image processor carries out model forward propagation, the first activation amount is asynchronously transmitted to the central processing unit; loading a first activation amount to the image processor from the central processing unit in a model back propagation process; obtaining the gradient of the model parameter calculated by the image processor based on the first activation amount, and asynchronously transmitting the gradient to the central processing unit; and determining an optimizer obtained based on gradient updating in the central processing unit, obtaining model parameters obtained by adjustment of the updated optimizer, and asynchronously transmitting the model parameters to the image processor. By adopting the method, the problems that a large model with limited video memory resources is difficult to train and the computing efficiency of a video card is low can be solved.

The invention discloses a multi-pose source fusion positioning method based on a factor graph, and the method comprises the steps: S1, building a factor graph: taking RTK state information at a fusion starting moment as a prior factor in a current state, and building the factor graph; s2, point cloud map relocation odometer information optimization: performing forward propagation by using IMU data, and establishing a state quantity error model; carrying out backward propagation on the original point cloud of the LiDAR by utilizing a predicted value of forward propagation so as to carry out distortion removal operation, and establishing an observation equation by utilizing the point cloud after distortion removal; then IEKF is used for updating; s3, optimizing wheel speed-IMU odometer information: optimizing rotation information provided by an IMU through error Kalman, and calculating a state quantity at an IMU data moment in combination with wheel speed data; and S4, multi-pose source fusion: carrying out cumulative multiplication on the RTK priori factor, the point cloud map relocation odometer factor and the wheel speed-IMU odometer factor to obtain a system state with the maximum probability. Through fusion optimization of data of a plurality of sensors, accurate autonomous positioning information of the mine card is obtained.

The invention belongs to the technical field of industrial equipment life prediction, and discloses an equipment life prediction method and system of a large language model based on semantic alignment, and the method comprises the steps: obtaining original multi-dimensional sensor time sequence data, and obtaining an embedded matrix after preprocessing; constructing a prompt text with domain semantics to obtain a natural language embedded representation; constructing a semantic text prototype, and realizing alignment of the embedding matrix and the semantic text prototype to obtain a patch embedding sequence; and splicing the patch embedding sequence and the natural language embedding representation, inputting the spliced patch embedding sequence and the natural language embedding representation into a pre-trained large language model for forward propagation, extracting hidden vectors output corresponding to the patch embedding sequence, splicing and flattening the hidden vectors into a single vector, and outputting to obtain an equipment life prediction result. According to the method, cross-modal knowledge learned by the LLM in large-scale pre-training and the powerful reasoning ability are fully utilized, and accurate prediction of the residual life of the equipment is achieved.

The invention discloses a fault early warning method and system for a battery pack based on data analysis, and the method comprises the steps: obtaining asynchronous sampling data of voltage, current and temperature parameters from a plurality of data sources, mapping the asynchronous sampling data to a preset time grid through employing a linear interpolation algorithm, and generating time series data; inputting the time sequence data into a long short-term memory network, and calculating through forward propagation to obtain a hidden state sequence containing time dependence characteristics; calculating a feature weight of the hidden state sequence by adopting an attention mechanism, and if the feature weight is greater than a preset threshold value, reserving corresponding features to form a simplified feature set; calculating posterior probability distribution by adopting a Bayesian reasoning algorithm based on the simplified feature set, and determining a fault occurrence probability; and according to the fault occurrence probability, a dynamic threshold adjustment algorithm is adopted to update an early warning threshold, and if the fault probability exceeds the current early warning threshold, a graded early warning signal is triggered. According to the invention, a powerful guarantee is provided for safe and stable operation of power equipment.

The invention discloses a segmentation federal learning model training method based on a heterogeneous system. The system comprises a plurality of clients, a central server and an edge server. The global model segments the client local model and the server local model based on each client device condition and deploys to corresponding clients and central servers, respectively. And in each round of training, the client performs parallel training, forwards propagates local data, outputs crushed data, transmits the crushed data to the server local model of the central server, continuously forwards propagates to obtain a loss value, and backwards propagates a gradient to update parameters of the server local model and the client local model. And each client transmits the updated model to the edge server, and the edge server and the central server complete parameter exchange of a public layer and then perform model gathering to generate a global client model and a global server model. The global client model distributes client local models to the clients according to each client break point. The method can adapt to equipment conditions of heterogeneous equipment, and the model training efficiency is improved.

The invention relates to the technical field of crane ship operation prediction, and discloses a BP neural network-based crane ship operability prediction method. The method comprises the following steps: collecting and preprocessing environmental data of a crane ship operation sea area, classifying and extracting features according to a preset operation state, and dividing a training test set; detecting sample data quality, and marking abnormal samples; using an input feature set of a non-abnormal sample and a historical operation state label matching result to train a model, and generating a BP neural network prediction model; the model parameters are optimized and corrected based on the incidence relation between the weight offset parameters and the prediction errors; and inputting real-time environment data, calculating an output probability along a network forward propagation path, determining the operability of the crane ship, and generating a prediction result. The method can effectively process the non-linear relationship between environmental factors, reduce abnormal data interference, improve the accuracy, objectivity and consistency of prediction, and provide a reliable basis for operation decision making of the crane ship.

The invention discloses a fine tuning system of a large-scale pre-training model in a federated learning environment and application thereof. The system comprises a local disturbance gradient estimation module, a differential privacy protection module and a global model aggregation and update module. The local disturbance gradient estimation module is used for calculating a global model loss value by combining forward propagation with a zero-order optimization method so as to estimate a gradient and realize all-parameter fine tuning; the differential privacy protection module performs differential privacy protection processing on the estimated disturbance gradient to prevent gradient information from leaking user sensitive data; and the global model aggregation and update module reconstructs a disturbance vector and completes global model update based on a random seed and a scalar gradient uploaded by a client. Compared with the prior art, on the premise of not depending on back propagation, all-parameter fine tuning of a large-scale pre-training model is achieved, data privacy is guaranteed, meanwhile, calculation and memory expenses are remarkably reduced, and the method is suitable for a resource-limited distributed calculation environment.

The invention provides a ship main engine power real-time optimization method based on a hybrid network model, and belongs to the technical field of ship energy saving.A hybrid neural network model is constructed, the hybrid neural network model is based on a physical information neural network, a KAN network is introduced to serve as a front-end network structure, and the power of a ship main engine is optimized in real time; the high-dimensional nonlinear mapping module is used for establishing high-dimensional nonlinear mapping from navigational speed, a ship type parameter matrix, propulsive efficiency, fuel conversion efficiency and environmental factors to the minimum power of a main engine; a composite loss function is adopted to train the hybrid neural network model, wherein the composite loss function is formed by weighted summation of mean square error loss, dynamics constraint loss, propulsive efficiency constraint loss and fuel consumption constraint loss; using the trained hybrid neural network model to receive ship operation parameters and environment parameters collected in real time, and outputting a minimum power prediction value of the ship main engine through one-time forward propagation calculation to realize real-time optimization of the main engine power.

The invention discloses a multi-mode industrial anomaly detection method fusing image masks and mixed experts. According to the method, an image mask technology and a hybrid expert method are fused to guide an industrial multi-modal pre-training model, and based on a multi-modal industrial data pre-training model, the pre-training large model can learn features which are difficult to express by a single mode. In the training stage, part of pixel blocks of a multi-modal input image are randomly masked by using a mask technology, the model can reconstruct visual features aligned with image texts of masked image blocks according to visible image blocks through training, and the model can learn low-level geometric structure features and high-level semantic information of the image from the model at the same time, so that the multi-modal input image is obtained. Therefore, most visual information required by downstream tasks is covered. An encoder part of an industrial large model adopts a hybrid expert structure to replace a forward propagation layer, and a small part of network layers can be dynamically activated during data processing, so that the model training and reasoning speed is increased while the model parameter quantity is kept.

The embodiment of the invention provides an assembly line optimization method and device for multi-modal large model training, and relates to the technical field of artificial intelligence. According to the method, each model slice of a to-be-trained multi-modal large model is deployed to different training devices, each training batch is divided into a plurality of micro-batches, for each training batch, training sequences corresponding to different training devices are determined according to a parallel scheduling strategy, and the training sequences are used for training the multi-modal large model. The training sequence comprises a forward propagation position, an input back propagation position and a weight back propagation position of each micro-batch, and finally, on each training device, a training process of a corresponding training batch is executed based on the training sequence until training is completed. A back propagation process is divided into back propagation of an input matrix and back propagation of a weight matrix, three calculation stages are jointly formed by the back propagation process and forward propagation, peak shifting calculation can be carried out, bubble time can be effectively filled, idle duration can be reduced, the bubble proportion of an assembly line can be remarkably reduced, and training throughput can be greatly improved.

The invention discloses a model training method and device, electronic equipment and a storage medium. Comprising the following steps: sampling an original video frame sequence based on a target training temporal-spatial resolution to obtain target training data; inputting the target training data into a teacher model, generating a target token set through dynamic token selection, and generating teacher training features through forward propagation; performing multi-scale cutting on the target token set according to a target self-attention weight of the teacher model to generate at least three student training masks with different token numbers; inputting the target training data and the different student training masks into a student model for forward propagation, and generating student training features; and performing alignment distillation on the student training features and the teacher training features to obtain a target student model. The defect of a video understanding model in downstream flexible reasoning is overcome, and dynamic token selection and multi-scale mask training under high temporal-spatial resolution are utilized, so that the model can obtain better performance under various downstream calculated amount limits.

The invention discloses a self-adaptive parameter high-efficiency fine tuning method, device and equipment and a storage medium, and the method comprises the steps: obtaining a parameter high-efficiency fine tuning super network which comprises a plurality of parameter fine tuning modules; obtaining training parameters, enabling the training parameters to participate in parameter training of a plurality of parameter fine tuning modules, obtaining a weight value of each parameter fine tuning module, obtaining a selection factor according to the weight value, and screening the plurality of parameter fine tuning modules in the parameter efficient fine tuning super network by using the selection factor to obtain a target parameter fine tuning module; and the selected parameter fine tuning module participates in forward propagation of a t ranform layer of the pre-training model, and a hidden state of an input sample output at the t ranform layer is modified. By constructing the parameter efficient fine tuning super network, the pre-training model adaptively selects the optimal parameter efficient fine tuning module according to the task, the capability of the large model is unlocked to the greatest extent, and the average performance of the deployed model on various medical tasks is improved.

The invention discloses a digital twin calibration framework construction method based on a Lyapunov strategy, and the method comprises the steps: constructing a digital twin model containing a proxy network to simulate the dynamic state of a system, and defining a state, an action space and a reward function through a Markov decision process. A constrained Lyapunov action-commentator (CLAC) algorithm is introduced, a strategy network and a Lyapunov network are optimized, and the algorithm can stably act under high noise and deviation. Real-time parameter optimization is realized by means of single-time neural network forward propagation, an experience playback pool and the like. Each network structure is clear, and a specific initialization and optimization method is adopted. According to the method, a calibration problem can be converted into a parameter tracking task, efficient training is performed under unmarked data, constraint requirements such as stability can be met, and the method is suitable for scenes such as industrial robot joint control.

The invention discloses a recalculation perception model splitting method and system oriented to assembly line parallel training, and belongs to the field of artificial intelligence of computer science. The method specifically comprises the steps that a dijkstra algorithm and two-layer binary search are used, and the minimum value of the sum of forward propagation time and reverse propagation time of assembly line microbatch under re-calculation perception model splitting is calculated; and an intermediate result obtained by a dijkstra algorithm in a search process is used to calculate and output an approximately optimal model splitting strategy for re-calculation perception. According to the method, the model splitting method and system are designed by utilizing the model characteristics of a Tasformer large model and the characteristics of an assembly line parallel mode, the model is non-uniformly split to each stage of the assembly line, so that the calculation load of each stage of the assembly line is balanced when re-calculation is started, and the calculation efficiency is improved. Therefore, the overall performance of large model pipeline parallel training and the utilization efficiency of cluster equipment are improved.

Disclosed in the present invention is an underwater acoustic target recognition method based on effective receptive field regulation. A proposed AEU-Net model has a plurality of resolution branches, with each resolution branch having independent convolution kernels, which adaptively adjust their sizes during training while aligning with their respective resolutions. The AEU-Net model comprises an ERF-Server, which can regulate an effective receptive field during forward propagation, and has two operations of increasing or decreasing the effective receptive field of a feature map of any designated module. In the present invention, acoustic physical information of an underwater acoustic target sample can be captured in a plurality of interaction dimensions, and feature fusion of information from effective receptive fields across a plurality of scales is performed, so as to adapt to sonar image targets of different sizes, thereby improving the accuracy and recognition speed in the recognition of an underwater acoustic target.

The invention is applied to the field of mechanical fault diagnosis signal processing, and particularly provides a pulse neural network mechanical fault diagnosis method based on STFT dimension transformation, and the method comprises the steps: collecting a one-dimensional mechanical vibration signal, carrying out the wavelet decomposition, carrying out the wavelet reconstruction of a low-frequency component and a denoised high-frequency component, and carrying out the wavelet reconstruction of the low-frequency component and the denoised high-frequency component; obtaining a denoised one-dimensional vibration signal; performing short-time Fourier transform, and converting the time-frequency two-dimensional matrix into a time-frequency two-dimensional matrix; inputting the time-frequency two-dimensional matrix into an improved HH threshold neuron model, carrying out Poisson sparse coding on the time-frequency two-dimensional matrix, and only carrying out pulse response on signal significant features; constructing a suprathreshold coding convolutional network with residual connection, inputting a sparse coding matrix, training by adopting an unsupervised learning rule based on STDP, and adaptively adjusting a network synaptic weight; and inputting to a trained above-threshold coding convolutional network, and obtaining pulse emission activity of neurons of an output layer through network forward propagation to determine a fault diagnosis result.

The invention relates to the technical field of large models, and discloses a large model adjustment method, device and equipment and a storage medium, the method comprises the following steps: carrying out segmentation deployment on a to-be-adjusted large model to obtain sub-models deployed at different end side equipment, each sub-model comprising different numbers of coding and decoding layers and adjustable adapters, a corresponding adjustable adapter is embedded behind each coding and decoding layer; updating the adjustable adapter layer by layer according to the training state of the to-be-adjusted large model to obtain a large model of the adjustable adapter embedded with updated configuration; and performing forward propagation calculation and backward propagation calculation on the large model of the adjustable adapter embedded with the updated configuration to obtain a large model after parameter adjustment. The parameter adjustment efficiency of the large model can be effectively improved, and equipment resources are saved.

The embodiment of the present invention discloses a method for predicting corneal reshaping parameters based on an artificial neural network, and relates to the technical field of orthokeratology lenses. The method comprises: obtaining an initial sample data set, deleting target samples that do not contain necessary parameters, and obtaining a basic sample data set; preprocessing the basic sample data set to obtain a final sample data set; training the initial artificial neural network model to obtain a preliminary model; using the preliminary model to forward propagate the input layer data of the basic sample data set to obtain a calculation result and compare it with the result of the basic sample to obtain a final sample data set, and using the final sample data set to train the preliminary model to obtain a prediction model; inputting actual eye detection data into the prediction model to obtain corresponding corneal reshaping parameters. The present invention splits the data to be predicted into multiple dimensions, constructs different prediction models according to each data to be predicted, and then performs distributed training and uses them in combination.

The invention relates to the field of federated learning security, and provides a privacy protection longitudinal federated learning method and system based on function encryption. According to the method, the problems that in a longitudinal federated learning scene, when privacy protection is achieved based on the cryptography technology, the calculation time is too long, the communication overhead is too large, and the model accuracy and the training overhead are difficult to balance are solved. The main scheme comprises system initialization, client forward propagation, server forward propagation, server backward propagation and client backward propagation. The method is used for privacy protection longitudinal federated learning, gradient leakage attacks and reverse derivation attacks based on model parameters are effectively resisted through a decentralized multi-client function encryption algorithm and a longitudinal federated learning framework, the calculation efficiency is improved, meanwhile, the model precision is guaranteed, the communication overhead is reduced, and the privacy of client sample data is enhanced.

The invention discloses a three-dimensional magnetotelluric inversion method, a three-dimensional magnetotelluric inversion device and three-dimensional magnetotelluric inversion equipment, and relates to the technical field of three-dimensional magnetotelluric inversion processing. A network model suitable for three-dimensional magnetotelluric inversion is constructed based on a deep learning technology, so that the input of the model is apparent resistivity and phase; the output is a three-dimensional resistivity distribution parameter of the to-be-measured area; a data-driven and physical-driven combined weighted loss function is designed and introduced into the network model, so that three-dimensional forward modeling operation of magnetotelluric is integrated into forward propagation of the network model, the network model can follow magnetotelluric physical constraints in the training process, and the inversion precision of the anomalous body is integrally improved.

The invention discloses a large language model weight and activation combined quantification method and system, and belongs to the technical field of model quantification. The method comprises the following steps: collecting and preprocessing a calibration set, inputting a large language model to execute forward propagation, and recording an activation matrix; for each embedding dimension, counting maximum activation absolute values of all lexical elements on the dimension; generating a global threshold value by combining a quantile statistical method with the global sensitivity coefficient, and determining that the dimension with the maximum activation absolute value exceeding the global threshold value in the dimensions is an outlier dimension; respectively designing scaling factors for the normal dimension and the outlier dimension, and generating a reconstruction weight matrix; using Bayesian-gradient joint optimization to reconstruct a truncation threshold value of the weight; calculating a scaling factor of the reconstructed weight matrix to obtain a reconstructed quantized weight matrix; quantizing the activation matrix of the current layer according to the embedding dimension by applying a scaling factor; and performing multiplication calculation with the reconstructed quantization weight matrix to obtain a multiplication output result in an integer domain, and then performing unified inverse quantization recovery.

The invention relates to the technical field of intelligent construction and digital twinning, and discloses an intelligent management platform system for seawall engineering construction whole-process data integration, and the system comprises a multi-source heterogeneous data collection module which is used for obtaining and standardizing construction data; the core data fusion and reasoning module is used for constructing a construction process causal atlas representing the relationship between elements and calculating the credibility of each state node through a multi-source evidence theory fusion engine; the intelligent application and evolution module is used for carrying out risk forward propagation early warning and root cause backward tracing diagnosis based on the construction process causal atlas; and the comprehensive visualization and decision support module is used for performing fusion display on the analysis result and the BIM / GIS model. According to the method, the causal knowledge graph and evidence fusion reasoning mechanism is constructed, the problems of data islands and information uncertainty are solved, active early warning and accurate traceability of construction risks and dynamic self-adaption of the system are achieved, and the intelligent management level of seawall engineering is remarkably improved.

The invention relates to an edge end spiking neural network compression and deployment method and system, and belongs to the technical field of machine learning, the edge end spiking neural network compression and deployment method performs forward propagation on an initial spiking neural network model based on acquired multi-modal data, obtaining a memory use state in a forward propagation process based on hardware sensing initialization, and performing dynamic sparsification on the initial pulse neural network model based on the memory use state to construct a sparsified pulse neural network; based on the acquired activation statistical information of the sparse spiking neural network, performing dynamic pruning on the sparse spiking neural network by adopting a pruning strategy based on neuron activeness so as to realize compression of a spiking neural network model; the compressed spiking neural network model is optimized according to the edge end configuration, and the optimized spiking neural network model is deployed to the edge end to execute the reasoning task, so that the storage requirement and the calculation complexity are reduced.

The invention discloses an electromechanical composite transmission system health state identification method based on dynamic kernel width, which belongs to the technical field of intelligent operation and maintenance of mechanical equipment and comprises the following steps: acquiring an electromechanical composite transmission system data set; building a health state identification network framework; parameters of each layer of the identification network framework are initialized, convolution kernel weight parameters of a sparse convolution layer are divided into a plurality of groups along an output channel, and each group corresponds to a search area; performing forward propagation and calculating loss; performing dynamic kernel wide layer convolution calculation optimization and sparse function training optimization; repeating forward propagation, loss calculation, sparse updating and gradient updating until the iteration task is finished, and obtaining a corresponding electromechanical composite transmission system health state identification model; and health state identification of the electromechanical composite transmission system is realized based on the model. According to the method, convolution calculation of a dynamic kernel width layer is guided by using a sparse function, the training complexity is reduced, and the balance of a large receptive field and low calculation complexity is realized.