939 results about "Forward propagation" patented technology

The invention discloses an engineering safety progress intelligent monitoring method based on multi-source data collaboration, which relates to the technical field of intelligent engineering monitoring, and comprises the following steps of: mapping multi-source engineering monitoring data into nodes and edges of a graph in real time by utilizing an incremental graph updating algorithm, generating a dynamic knowledge graph, and generating a dynamic mapping result; performing graph traversal on the dynamic knowledge graph through an association subgraph extraction algorithm, extracting a security event matrix and a project progress matrix, calculating an SPI index value by using a dynamic weighted fusion algorithm, synchronously performing multi-threshold interval grading on the SPI index value, generating an SPI early warning level, performing state coding on the SPI early warning level, and generating a comprehensive feature vector; according to the method, data of different types can be effectively integrated and a basis is provided for formulating a targeted engineering safety progress solution through an incremental graph updating algorithm and a Bayesian causal graph model, and node probability distribution characteristics are extracted by using a forward propagation layer. The method is advantaged in that the incremental graph updating algorithm and the Bayesian causal graph model are utilized to effectively integrate data of different types and provide a basis for formulating a targeted engineering safety progress solution.

The invention relates to a training method, a data processing method, electronic equipment and a computer readable storage medium, and relates to the technical field of computers. The training method comprises the steps that a training sample is input into a to-be-trained model, forward propagation is executed, output of the to-be-trained model is obtained, the to-be-trained model comprises a plurality of expert layers, and a specific activation value is not stored in the forward propagation process; determining the value of a loss function according to the output of the to-be-trained model; according to the value of the loss function and the loss function, back propagation is executed, in the back propagation process of each expert layer, a recalculation task of a specific activation value and a first communication task are executed in an overlapping mode, and for each expert layer, the first communication task comprises a communication task of data interaction among multiple devices corresponding to the expert layer.

The invention provides a deep learning large model training method and system for heterogeneous equipment, belongs to the technical field of model training, and aims to solve the problem that a heterogeneous GPU cluster cannot be effectively utilized when a traditional scheme is used for training a deep learning large model. Different network layers of a deep learning large model to be trained are divided into a plurality of stages, forward propagation and back propagation calculation of all network layers in each stage is executed by independent virtual equipment, and meanwhile, utilization of GPU resources of different structures is coordinated in combination with a proposed hybrid parallel training strategy, so that efficient model training is realized.

The invention relates to a resource-limited large model heterogeneous training method, computer equipment and a storage medium, and the method comprises the steps: determining a first activation amount according to a dependency relationship between calculated amounts corresponding to the activation amounts and / or the calculated amount and the storage amount of each activation amount in the activation amounts generated when an image processor carries out model forward propagation, the first activation amount is asynchronously transmitted to the central processing unit; loading a first activation amount to the image processor from the central processing unit in a model back propagation process; obtaining the gradient of the model parameter calculated by the image processor based on the first activation amount, and asynchronously transmitting the gradient to the central processing unit; and determining an optimizer obtained based on gradient updating in the central processing unit, obtaining model parameters obtained by adjustment of the updated optimizer, and asynchronously transmitting the model parameters to the image processor. By adopting the method, the problems that a large model with limited video memory resources is difficult to train and the computing efficiency of a video card is low can be solved.

The invention belongs to the technical field of industrial equipment life prediction, and discloses an equipment life prediction method and system of a large language model based on semantic alignment, and the method comprises the steps: obtaining original multi-dimensional sensor time sequence data, and obtaining an embedded matrix after preprocessing; constructing a prompt text with domain semantics to obtain a natural language embedded representation; constructing a semantic text prototype, and realizing alignment of the embedding matrix and the semantic text prototype to obtain a patch embedding sequence; and splicing the patch embedding sequence and the natural language embedding representation, inputting the spliced patch embedding sequence and the natural language embedding representation into a pre-trained large language model for forward propagation, extracting hidden vectors output corresponding to the patch embedding sequence, splicing and flattening the hidden vectors into a single vector, and outputting to obtain an equipment life prediction result. According to the method, cross-modal knowledge learned by the LLM in large-scale pre-training and the powerful reasoning ability are fully utilized, and accurate prediction of the residual life of the equipment is achieved.

The invention discloses a fault early warning method and system for a battery pack based on data analysis, and the method comprises the steps: obtaining asynchronous sampling data of voltage, current and temperature parameters from a plurality of data sources, mapping the asynchronous sampling data to a preset time grid through employing a linear interpolation algorithm, and generating time series data; inputting the time sequence data into a long short-term memory network, and calculating through forward propagation to obtain a hidden state sequence containing time dependence characteristics; calculating a feature weight of the hidden state sequence by adopting an attention mechanism, and if the feature weight is greater than a preset threshold value, reserving corresponding features to form a simplified feature set; calculating posterior probability distribution by adopting a Bayesian reasoning algorithm based on the simplified feature set, and determining a fault occurrence probability; and according to the fault occurrence probability, a dynamic threshold adjustment algorithm is adopted to update an early warning threshold, and if the fault probability exceeds the current early warning threshold, a graded early warning signal is triggered. According to the invention, a powerful guarantee is provided for safe and stable operation of power equipment.

The invention relates to the technical field of crane ship operation prediction, and discloses a BP neural network-based crane ship operability prediction method. The method comprises the following steps: collecting and preprocessing environmental data of a crane ship operation sea area, classifying and extracting features according to a preset operation state, and dividing a training test set; detecting sample data quality, and marking abnormal samples; using an input feature set of a non-abnormal sample and a historical operation state label matching result to train a model, and generating a BP neural network prediction model; the model parameters are optimized and corrected based on the incidence relation between the weight offset parameters and the prediction errors; and inputting real-time environment data, calculating an output probability along a network forward propagation path, determining the operability of the crane ship, and generating a prediction result. The method can effectively process the non-linear relationship between environmental factors, reduce abnormal data interference, improve the accuracy, objectivity and consistency of prediction, and provide a reliable basis for operation decision making of the crane ship.

The invention discloses a fine tuning system of a large-scale pre-training model in a federated learning environment and application thereof. The system comprises a local disturbance gradient estimation module, a differential privacy protection module and a global model aggregation and update module. The local disturbance gradient estimation module is used for calculating a global model loss value by combining forward propagation with a zero-order optimization method so as to estimate a gradient and realize all-parameter fine tuning; the differential privacy protection module performs differential privacy protection processing on the estimated disturbance gradient to prevent gradient information from leaking user sensitive data; and the global model aggregation and update module reconstructs a disturbance vector and completes global model update based on a random seed and a scalar gradient uploaded by a client. Compared with the prior art, on the premise of not depending on back propagation, all-parameter fine tuning of a large-scale pre-training model is achieved, data privacy is guaranteed, meanwhile, calculation and memory expenses are remarkably reduced, and the method is suitable for a resource-limited distributed calculation environment.

The invention provides a ship main engine power real-time optimization method based on a hybrid network model, and belongs to the technical field of ship energy saving.A hybrid neural network model is constructed, the hybrid neural network model is based on a physical information neural network, a KAN network is introduced to serve as a front-end network structure, and the power of a ship main engine is optimized in real time; the high-dimensional nonlinear mapping module is used for establishing high-dimensional nonlinear mapping from navigational speed, a ship type parameter matrix, propulsive efficiency, fuel conversion efficiency and environmental factors to the minimum power of a main engine; a composite loss function is adopted to train the hybrid neural network model, wherein the composite loss function is formed by weighted summation of mean square error loss, dynamics constraint loss, propulsive efficiency constraint loss and fuel consumption constraint loss; using the trained hybrid neural network model to receive ship operation parameters and environment parameters collected in real time, and outputting a minimum power prediction value of the ship main engine through one-time forward propagation calculation to realize real-time optimization of the main engine power.

The invention discloses a multi-mode industrial anomaly detection method fusing image masks and mixed experts. According to the method, an image mask technology and a hybrid expert method are fused to guide an industrial multi-modal pre-training model, and based on a multi-modal industrial data pre-training model, the pre-training large model can learn features which are difficult to express by a single mode. In the training stage, part of pixel blocks of a multi-modal input image are randomly masked by using a mask technology, the model can reconstruct visual features aligned with image texts of masked image blocks according to visible image blocks through training, and the model can learn low-level geometric structure features and high-level semantic information of the image from the model at the same time, so that the multi-modal input image is obtained. Therefore, most visual information required by downstream tasks is covered. An encoder part of an industrial large model adopts a hybrid expert structure to replace a forward propagation layer, and a small part of network layers can be dynamically activated during data processing, so that the model training and reasoning speed is increased while the model parameter quantity is kept.

The embodiment of the invention provides an assembly line optimization method and device for multi-modal large model training, and relates to the technical field of artificial intelligence. According to the method, each model slice of a to-be-trained multi-modal large model is deployed to different training devices, each training batch is divided into a plurality of micro-batches, for each training batch, training sequences corresponding to different training devices are determined according to a parallel scheduling strategy, and the training sequences are used for training the multi-modal large model. The training sequence comprises a forward propagation position, an input back propagation position and a weight back propagation position of each micro-batch, and finally, on each training device, a training process of a corresponding training batch is executed based on the training sequence until training is completed. A back propagation process is divided into back propagation of an input matrix and back propagation of a weight matrix, three calculation stages are jointly formed by the back propagation process and forward propagation, peak shifting calculation can be carried out, bubble time can be effectively filled, idle duration can be reduced, the bubble proportion of an assembly line can be remarkably reduced, and training throughput can be greatly improved.

The invention discloses a model training method and device, electronic equipment and a storage medium. Comprising the following steps: sampling an original video frame sequence based on a target training temporal-spatial resolution to obtain target training data; inputting the target training data into a teacher model, generating a target token set through dynamic token selection, and generating teacher training features through forward propagation; performing multi-scale cutting on the target token set according to a target self-attention weight of the teacher model to generate at least three student training masks with different token numbers; inputting the target training data and the different student training masks into a student model for forward propagation, and generating student training features; and performing alignment distillation on the student training features and the teacher training features to obtain a target student model. The defect of a video understanding model in downstream flexible reasoning is overcome, and dynamic token selection and multi-scale mask training under high temporal-spatial resolution are utilized, so that the model can obtain better performance under various downstream calculated amount limits.

The invention discloses a digital twin calibration framework construction method based on a Lyapunov strategy, and the method comprises the steps: constructing a digital twin model containing a proxy network to simulate the dynamic state of a system, and defining a state, an action space and a reward function through a Markov decision process. A constrained Lyapunov action-commentator (CLAC) algorithm is introduced, a strategy network and a Lyapunov network are optimized, and the algorithm can stably act under high noise and deviation. Real-time parameter optimization is realized by means of single-time neural network forward propagation, an experience playback pool and the like. Each network structure is clear, and a specific initialization and optimization method is adopted. According to the method, a calibration problem can be converted into a parameter tracking task, efficient training is performed under unmarked data, constraint requirements such as stability can be met, and the method is suitable for scenes such as industrial robot joint control.

The invention discloses a recalculation perception model splitting method and system oriented to assembly line parallel training, and belongs to the field of artificial intelligence of computer science. The method specifically comprises the steps that a dijkstra algorithm and two-layer binary search are used, and the minimum value of the sum of forward propagation time and reverse propagation time of assembly line microbatch under re-calculation perception model splitting is calculated; and an intermediate result obtained by a dijkstra algorithm in a search process is used to calculate and output an approximately optimal model splitting strategy for re-calculation perception. According to the method, the model splitting method and system are designed by utilizing the model characteristics of a Tasformer large model and the characteristics of an assembly line parallel mode, the model is non-uniformly split to each stage of the assembly line, so that the calculation load of each stage of the assembly line is balanced when re-calculation is started, and the calculation efficiency is improved. Therefore, the overall performance of large model pipeline parallel training and the utilization efficiency of cluster equipment are improved.

Disclosed in the present invention is an underwater acoustic target recognition method based on effective receptive field regulation. A proposed AEU-Net model has a plurality of resolution branches, with each resolution branch having independent convolution kernels, which adaptively adjust their sizes during training while aligning with their respective resolutions. The AEU-Net model comprises an ERF-Server, which can regulate an effective receptive field during forward propagation, and has two operations of increasing or decreasing the effective receptive field of a feature map of any designated module. In the present invention, acoustic physical information of an underwater acoustic target sample can be captured in a plurality of interaction dimensions, and feature fusion of information from effective receptive fields across a plurality of scales is performed, so as to adapt to sonar image targets of different sizes, thereby improving the accuracy and recognition speed in the recognition of an underwater acoustic target.

The invention is applied to the field of mechanical fault diagnosis signal processing, and particularly provides a pulse neural network mechanical fault diagnosis method based on STFT dimension transformation, and the method comprises the steps: collecting a one-dimensional mechanical vibration signal, carrying out the wavelet decomposition, carrying out the wavelet reconstruction of a low-frequency component and a denoised high-frequency component, and carrying out the wavelet reconstruction of the low-frequency component and the denoised high-frequency component; obtaining a denoised one-dimensional vibration signal; performing short-time Fourier transform, and converting the time-frequency two-dimensional matrix into a time-frequency two-dimensional matrix; inputting the time-frequency two-dimensional matrix into an improved HH threshold neuron model, carrying out Poisson sparse coding on the time-frequency two-dimensional matrix, and only carrying out pulse response on signal significant features; constructing a suprathreshold coding convolutional network with residual connection, inputting a sparse coding matrix, training by adopting an unsupervised learning rule based on STDP, and adaptively adjusting a network synaptic weight; and inputting to a trained above-threshold coding convolutional network, and obtaining pulse emission activity of neurons of an output layer through network forward propagation to determine a fault diagnosis result.

The invention relates to the technical field of large models, and discloses a large model adjustment method, device and equipment and a storage medium, the method comprises the following steps: carrying out segmentation deployment on a to-be-adjusted large model to obtain sub-models deployed at different end side equipment, each sub-model comprising different numbers of coding and decoding layers and adjustable adapters, a corresponding adjustable adapter is embedded behind each coding and decoding layer; updating the adjustable adapter layer by layer according to the training state of the to-be-adjusted large model to obtain a large model of the adjustable adapter embedded with updated configuration; and performing forward propagation calculation and backward propagation calculation on the large model of the adjustable adapter embedded with the updated configuration to obtain a large model after parameter adjustment. The parameter adjustment efficiency of the large model can be effectively improved, and equipment resources are saved.

The embodiment of the present invention discloses a method for predicting corneal reshaping parameters based on an artificial neural network, and relates to the technical field of orthokeratology lenses. The method comprises: obtaining an initial sample data set, deleting target samples that do not contain necessary parameters, and obtaining a basic sample data set; preprocessing the basic sample data set to obtain a final sample data set; training the initial artificial neural network model to obtain a preliminary model; using the preliminary model to forward propagate the input layer data of the basic sample data set to obtain a calculation result and compare it with the result of the basic sample to obtain a final sample data set, and using the final sample data set to train the preliminary model to obtain a prediction model; inputting actual eye detection data into the prediction model to obtain corresponding corneal reshaping parameters. The present invention splits the data to be predicted into multiple dimensions, constructs different prediction models according to each data to be predicted, and then performs distributed training and uses them in combination.

The invention relates to the field of federated learning security, and provides a privacy protection longitudinal federated learning method and system based on function encryption. According to the method, the problems that in a longitudinal federated learning scene, when privacy protection is achieved based on the cryptography technology, the calculation time is too long, the communication overhead is too large, and the model accuracy and the training overhead are difficult to balance are solved. The main scheme comprises system initialization, client forward propagation, server forward propagation, server backward propagation and client backward propagation. The method is used for privacy protection longitudinal federated learning, gradient leakage attacks and reverse derivation attacks based on model parameters are effectively resisted through a decentralized multi-client function encryption algorithm and a longitudinal federated learning framework, the calculation efficiency is improved, meanwhile, the model precision is guaranteed, the communication overhead is reduced, and the privacy of client sample data is enhanced.

The invention discloses a large language model weight and activation combined quantification method and system, and belongs to the technical field of model quantification. The method comprises the following steps: collecting and preprocessing a calibration set, inputting a large language model to execute forward propagation, and recording an activation matrix; for each embedding dimension, counting maximum activation absolute values of all lexical elements on the dimension; generating a global threshold value by combining a quantile statistical method with the global sensitivity coefficient, and determining that the dimension with the maximum activation absolute value exceeding the global threshold value in the dimensions is an outlier dimension; respectively designing scaling factors for the normal dimension and the outlier dimension, and generating a reconstruction weight matrix; using Bayesian-gradient joint optimization to reconstruct a truncation threshold value of the weight; calculating a scaling factor of the reconstructed weight matrix to obtain a reconstructed quantized weight matrix; quantizing the activation matrix of the current layer according to the embedding dimension by applying a scaling factor; and performing multiplication calculation with the reconstructed quantization weight matrix to obtain a multiplication output result in an integer domain, and then performing unified inverse quantization recovery.

The invention relates to the technical field of intelligent construction and digital twinning, and discloses an intelligent management platform system for seawall engineering construction whole-process data integration, and the system comprises a multi-source heterogeneous data collection module which is used for obtaining and standardizing construction data; the core data fusion and reasoning module is used for constructing a construction process causal atlas representing the relationship between elements and calculating the credibility of each state node through a multi-source evidence theory fusion engine; the intelligent application and evolution module is used for carrying out risk forward propagation early warning and root cause backward tracing diagnosis based on the construction process causal atlas; and the comprehensive visualization and decision support module is used for performing fusion display on the analysis result and the BIM / GIS model. According to the method, the causal knowledge graph and evidence fusion reasoning mechanism is constructed, the problems of data islands and information uncertainty are solved, active early warning and accurate traceability of construction risks and dynamic self-adaption of the system are achieved, and the intelligent management level of seawall engineering is remarkably improved.

The invention relates to an edge end spiking neural network compression and deployment method and system, and belongs to the technical field of machine learning, the edge end spiking neural network compression and deployment method performs forward propagation on an initial spiking neural network model based on acquired multi-modal data, obtaining a memory use state in a forward propagation process based on hardware sensing initialization, and performing dynamic sparsification on the initial pulse neural network model based on the memory use state to construct a sparsified pulse neural network; based on the acquired activation statistical information of the sparse spiking neural network, performing dynamic pruning on the sparse spiking neural network by adopting a pruning strategy based on neuron activeness so as to realize compression of a spiking neural network model; the compressed spiking neural network model is optimized according to the edge end configuration, and the optimized spiking neural network model is deployed to the edge end to execute the reasoning task, so that the storage requirement and the calculation complexity are reduced.

The invention relates to a neural network global one-time structured pruning method, system and device and a medium, and the method comprises the steps: carrying out the parallel capturing of the input activation tensors of all target layers in a to-be-pruned neural network through a calibration data set through single-time forward propagation; according to an input activation tensor, synchronously calculating a difference entropy index and amplitude response intensity for an intermediate neuron weight group of each target layer, and performing normalization and fusion to form a static global importance map; determining an importance threshold according to a preset pruning rate, and generating a global to-be-pruned index set of which the mixed importance score is lower than the importance threshold at one time based on the global importance map; and on the basis of the index set, performing one-time physical structured pruning on the weight matrix of each target layer. Therefore, the static global importance map is generated through single forward propagation and parallel capture of the activation tensor, and maskless one-time pruning is completed through physical structured pruning.

A system includes a neural network architecture. It a new type of neural network able to process statically mapped as well as temporally sequenced information with much better power utilization, data requirements and operational efficiencies. Unlike prior artificial neural network approaches, the present invention includes uniquely defined sets of relationships. The unique use of non-linear input-output mapping functions combined with a time-variant pilot function, and a deterministically bounded, fully-differentiable, nonlinear resonance field subsystem allows the present invention to be readily deployed to work with virtually any neural network architecture / implementation including photonic, opto-acoustic or other variants. This dramatically reduces the size and complexity of virtually any neural network architecture because it offloads what would otherwise need to be done in the form of back / forward propagation trained weights and biases to much simpler, more scalable differentiable input / output mapping functions.

A set of features of a training document are identified in a training document for training a machine learning model. A subset of the features is selected to be omitted from a training forward propagation. As a result of omitting the subset of the set of features, a different subset of the set of features is used to train the machine learning model to classify documents and distinguish between an out-of-domain document and in-domain document.

The invention provides a model training method and device, equipment and a storage medium, and relates to the technical field of computers, in particular to the technical field of neural network models and model training. The specific implementation scheme is as follows: a calculation unit executes quantization matrix multiplication based on Hadamard pre-transformation on an activation tensor and a weight tensor of a target model stored in a memory so as to generate an output tensor of a linear layer based on a low-precision tensor with smaller data bit width; using the output tensor and a subsequent network layer of the target model to complete forward propagation so as to obtain a loss value; and according to the loss value, updating model parameters of the target model stored in a memory through a back propagation algorithm. By means of the technical scheme, on the premise that the model training precision is guaranteed, memory resource occupation and the calculation amount in the calculation process can be remarkably reduced, and the training cost is reduced.

The invention discloses a deep learning table recognition model evolution-oriented regression defect relieving method, which comprises the following steps of: firstly, screening new features by applying a CONMI evaluation function in combination with specific hyper-parameters in a feature selection stage, and taking a new feature set obtained by combining the new features with an old feature set as model training data to realize contribution degree control-based retraining; on one hand, the feature contribution degree is calculated through the expected gradient, on the other hand, an overall loss function containing cosine similarity and a cross entropy loss function is designed to stabilize the old feature contribution degree and learn new features, and a model is trained through forward propagation and back propagation processes based on a new feature set; and under the condition that the accuracy rate of the table recognition model of the new version meets the condition, constructing a feature evolution scene, realizing a model evolution process according to a regression mitigation strategy, and taking the accuracy rate of the table recognition model of the new version as an evaluation index to obtain an evolution process regression defect mitigation strategy. The method is beneficial to improving the quality and reliability of a deep learning table recognition model.

The invention relates to the technical field of elevator fault diagnosis, and discloses an elevator fault diagnosis method and system.The elevator fault diagnosis method comprises the steps that elevator operation data are collected, operation stages are divided for the elevator operation process according to the elevator operation data, and the residual error of the elevator operation data is calculated in the operation stages; based on an elevator structure, a causal chain graph reflecting the relation between components is established through fault mode analysis, and a direct upstream node set of downstream nodes in the causal chain graph serves as a parent set of the downstream nodes; in the operation stage, performing block replacement processing for keeping time sequence characteristics on an upstream node residual sequence of each edge in the causal chain graph to generate a contrast residual sequence, and calculating the propagation intensity of the edge according to the contrast residual sequence; effective propagation paths are screened according to the propagation intensity of the edges, the effective propagation paths are sorted to obtain candidate nodes, residual errors of the candidate nodes are set to be zero through virtual pinch-off, forward propagation is carried out through a semi-physical model, and fault nodes are determined in the candidate nodes based on forward propagation results.

The invention belongs to the field of image processing, and discloses a medical image segmentation method based on multi-branch distillation, which comprises the following steps: constructing and training a multi-branch collaborative distillation image segmentation model based on uncertainty perception, and establishing a network architecture comprising a public encoder, a first decoder and a second decoder, including a guiding decoder and a guided decoder; dropout disturbance is differentiated on the output of the public encoder, multiple features are generated, multi-branch forward propagation is executed, and a virtual label is generated through fusion of a CTF module; calculating supervision loss and distribution alignment loss of the original feature map through a guided branch; performing multi-level consistency constraint on the output of the disturbance characteristic graph by the guided model to realize multi-view structure consistency, automatically balancing parameter weights through an adaptive task balancing mechanism, and constructing a total loss function; and obtaining a to-be-segmented medical image, and inputting the to-be-segmented medical image into the trained image segmentation model to obtain a segmentation result. According to the invention, high-quality medical image segmentation under a low marking rate can be realized.

The invention provides a tensor segmentation and mapping method of a neural network operator on a wafer chip, and the method comprises the steps: expanding a forward calculation graph into a training calculation graph containing forward propagation, back propagation and gradient updating, and employing different operators in the calculation of these stages; carrying out dimension segmentation on the input tensor of each operator in the training calculation graph to generate a plurality of global candidate segmentation strategies; aiming at each candidate segmentation strategy, enumerating feasible resource mapping strategies of all operators, and combining to generate candidate design points; and screening an optimal design point through cost evaluation, and outputting a corresponding candidate segmentation strategy and a resource mapping strategy. According to the method, forward and backward operators in a neural network training process are decoupled through expansion of a calculation graph, and a segmentation scheme for avoiding tensor copying in forward and backward calculation in the training process and a resource mapping strategy corresponding to the segmentation scheme are explored, so that tensor copying is avoided, global resource occupation is reduced, and the training efficiency is improved. Therefore, the same wafer-level chip can support larger model training.