250 results about "Backward propagation" patented technology

The invention discloses a multi-pose source fusion positioning method based on a factor graph, and the method comprises the steps: S1, building a factor graph: taking RTK state information at a fusion starting moment as a prior factor in a current state, and building the factor graph; s2, point cloud map relocation odometer information optimization: performing forward propagation by using IMU data, and establishing a state quantity error model; carrying out backward propagation on the original point cloud of the LiDAR by utilizing a predicted value of forward propagation so as to carry out distortion removal operation, and establishing an observation equation by utilizing the point cloud after distortion removal; then IEKF is used for updating; s3, optimizing wheel speed-IMU odometer information: optimizing rotation information provided by an IMU through error Kalman, and calculating a state quantity at an IMU data moment in combination with wheel speed data; and S4, multi-pose source fusion: carrying out cumulative multiplication on the RTK priori factor, the point cloud map relocation odometer factor and the wheel speed-IMU odometer factor to obtain a system state with the maximum probability. Through fusion optimization of data of a plurality of sensors, accurate autonomous positioning information of the mine card is obtained.

The invention relates to the technical field of large models, and discloses a large model adjustment method, device and equipment and a storage medium, the method comprises the following steps: carrying out segmentation deployment on a to-be-adjusted large model to obtain sub-models deployed at different end side equipment, each sub-model comprising different numbers of coding and decoding layers and adjustable adapters, a corresponding adjustable adapter is embedded behind each coding and decoding layer; updating the adjustable adapter layer by layer according to the training state of the to-be-adjusted large model to obtain a large model of the adjustable adapter embedded with updated configuration; and performing forward propagation calculation and backward propagation calculation on the large model of the adjustable adapter embedded with the updated configuration to obtain a large model after parameter adjustment. The parameter adjustment efficiency of the large model can be effectively improved, and equipment resources are saved.

The invention discloses a classification model training method based on big data distributed computing, and relates to the technical field of big data processing and machine learning, and the method comprises the steps: S1, obtaining communication delay data between distributed computing nodes, and analyzing delay characteristics to generate regulation and control parameters, the method comprises the steps of S1, dynamically determining the batch size of each training batch based on regulation and control parameters, and ensuring the optimization of the synchronization efficiency, S3, distributing the adjusted batch size to each node, and executing the forward and reverse propagation calculation process of the model, and S4, collecting the calculation result of each node to perform parameter aggregation updating, and completing the iterative training of the classification model. According to the classification model training method based on big data distributed computing, the batch size is regulated and controlled according to the communication delay between the nodes so as to solve the problem that the distributed synchronization efficiency is low.

The invention relates to the field of federated learning security, and provides a privacy protection longitudinal federated learning method and system based on function encryption. According to the method, the problems that in a longitudinal federated learning scene, when privacy protection is achieved based on the cryptography technology, the calculation time is too long, the communication overhead is too large, and the model accuracy and the training overhead are difficult to balance are solved. The main scheme comprises system initialization, client forward propagation, server forward propagation, server backward propagation and client backward propagation. The method is used for privacy protection longitudinal federated learning, gradient leakage attacks and reverse derivation attacks based on model parameters are effectively resisted through a decentralized multi-client function encryption algorithm and a longitudinal federated learning framework, the calculation efficiency is improved, meanwhile, the model precision is guaranteed, the communication overhead is reduced, and the privacy of client sample data is enhanced.

The invention discloses a power load prediction model establishment method based on a convolutional bidirectional long-short-term memory network and an attention mechanism, and the method comprises the steps: carrying out the full feature extraction of original complex power load data through a one-dimensional convolutional network; the factors influencing the load and the characteristics of trend, periodicity, randomness and the like of load change can be more fully mastered by considering the multivariate characteristics; compared with the traditional LSTM, the bidirectional structure of the BiLSTM can more comprehensively capture the bidirectional dependency relationship in the load sequence data through forward propagation and backward propagation, and the understanding of the model on the data is improved. Besides, an attention mechanism widely applied in the field of natural language processing is applied to research on multi-element short-term power load prediction, key information in a load sequence can be focused by introducing the attention mechanism, and the prediction precision is further improved; experiments are carried out on a real multivariate load data set, and the precision of multivariate short-term load prediction can be effectively improved by adding the one-dimensional convolutional network and the attention mechanism.

The invention relates to a quantitative perception training method and device of a neural network model, electronic equipment and a storage medium. The method comprises the steps of obtaining a to-be-trained first neural network model; an operator pair and a non-module operator in the first neural network model are identified, the non-module operator is an operation which is not realized based on a module class, and the operator pair comprises a convolution operator and a batch normalization operator which are connected; the non-module operators are packaged into module operators, the operator pairs are packaged into new convolution operators, a second neural network model is obtained, the new convolution operators run the fusion process of the convolution operators and batch normalization operators during forward propagation, and the parameters of the convolution operators and the batch normalization operators are updated during back propagation; and performing quantitative perception training on the second neural network model. By adopting the method, the reasoning precision of the quantitative model can be guaranteed, and the deployment efficiency of the quantitative model is improved.

The invention relates to the technical field of deep learning, and discloses a fine-grained pipeline scheduling method and device based on cluster node memory difference perception. The method comprises the following steps: estimating the ratio of the peak memory occupancy to the memory capacity of each GPU node, comparing the ratio with a preset threshold value, and selecting a re-calculation strategy and a back propagation segmentation strategy: by taking minimization of the end-to-end training time of a model as a target, scheduling and modeling micro-batch data as a flow shop problem, analyzing the dependency constraint of each flow line stage, and calculating the flow shop problem; comprising forward-back propagation sequence dependence, stage dependence, operation dependence and memory limitation, and generating a micro-batch operation sequence scheduling scheme meeting the memory capacity limitation; according to the scheduling scheme, an execution sequence queue is created by executing sorting, and the execution time sequence of calculation blocks, communication blocks and re-calculation operation is dynamically coordinated. According to the method, the GPU equipment utilization rate can be remarkably improved, and the end-to-end training completion time of the model is shortened.

The invention discloses a method for automatically searching an optimal modal fusion strategy. The method comprises the following steps: S1, constructing a multi-task learning loss function L (theta); s2, inner layer optimization: using a current fusion strategy parameter alpha, executing forward and backward propagation on training data, updating a model parameter theta, and minimizing a training loss function Ltr which comprises L (theta); s3, outer layer optimization: evaluating performance on verification data by using the updated model parameter theta, calculating a gradient about alpha, updating a fusion strategy parameter alpha, and minimizing a verification loss function Lval; and S4, the steps S2 and S3 are repeated until the preset number of iterations is reached or the model performance is not improved any more, and the obtained fusion strategy parameter is the optimal modal fusion strategy. According to the scheme, the fusion strategy can be dynamically adjusted according to different data set characteristics and modal missing conditions, so that the potential of the model in multi-modal learning is exerted to the greatest extent.

A method of accelerating a training process of a neural network includes acquiring activations used in the training process and a bit-vector corresponding to the activations, selecting activations requiring an operation from among the acquired activations by using the bit-vector, and performing backward propagation using the selected activations and filters corresponding to the selected activations.

The invention belongs to the technical field of target detection, and particularly relates to a teacher tensor knowledge distillation-driven anti-violation target detection method in an electric power safety supervision scene, and the method comprises the following steps: inputting image data to a student model and a pre-trained teacher model; performing forward propagation on the student model to obtain a student prediction tensor; associating differences between the real tags based on the student prediction tensor and the input image data; acquiring an original teacher output tensor by using the same input image data by calling a bottom layer forward propagation method of a teacher model; calculating knowledge distillation loss based on the difference between the student prediction tensor and the original teacher output tensor; the standard detection loss and the knowledge distillation loss are combined to form total loss; and performing back propagation updating on parameters of the student model based on the total loss so as to complete model training and real-time target detection and detection method updating optimization. According to the invention, the performance of the student model can be significantly improved, and the high efficiency of the model is maintained.

An apparatus includes circuitry for a neural network that is configured to perform forward propagation neural network operations on floating point numbers having a first n-bit floating point format. The first n-bit floating point format has a configuration consisting of a sign bit, m exponent bits and p mantissa bits where m is greater than p. The circuitry is further configured to perform backward propagation neural network operations on floating point numbers having a second n-bit floating point format that is different than the first n-bit floating point format. The second n-bit floating point format has a configuration consisting of a sign bit, q exponent bits and r mantissa bits where q is greater than m and r is less than p.

Embodiments of the invention disclose a slow node detection method and apparatus, a device, a storage medium and a program product. The method comprises the steps of obtaining respective first duration data of a plurality of processing nodes participating in distributed model training; the first duration data represents the calculation time consumption of the corresponding processing node in the current iteration; based on a parallel strategy trained by the distributed model, grouping the first duration data to obtain a first duration set of multiple groups; the parallel strategy represents an association relationship of the plurality of processing nodes in a forward and backward propagation process and training tasks undertaken by the plurality of processing nodes; and based on the respective probability density function and a first threshold value of the plurality of groups, respectively analyzing and processing the first duration sets of the plurality of groups to obtain slow nodes in the plurality of processing nodes. Thus, the slow nodes in distributed model training can be intelligently and accurately detected, the efficiency and accuracy of slow node detection are improved, and the method has high compatibility.

Some embodiments of the present application provide a forward-propagation-only (FP-only) method of training a DNN model. Such methods result in a trained DNN model whose performance comparable to a DNN model trained using bidirectional training methods. The FP-only method for training a DNN model may operate without employing the known chain rule. The chain rule is employed when computing gradients for a backward propagation in a bidirectional method. The FP-only method may allow for the computations and updates to the parameters for each layer of the DNN model to be performed in parallel. The FP-only methods for training a DNN model use stochastic gradient descent and the FP-only method for training a DNN model still involves computing gradients. However, the FP-only methods of training a DNN model allow for computing of gradients without the chain rule.

The invention relates to a model training task execution method, a graphics processing unit and a model training system.The model training task execution method comprises the steps that when attention calculation of model forward propagation is executed on a current text segment, the current text segment is subjected to attention calculation; sequentially loading key values corresponding to all the preamble segments from the central processing unit to carry out attention calculation on the activation values; meanwhile, the key value pair activation value corresponding to the current text segment is unloaded to the central processing unit; when the attention calculation of model back propagation is executed on the current text segment, loading a forward propagation output result and a key value pair activation value corresponding to the current text segment from a central processing unit, and loading the attention activation value corresponding to each preorder text segment in sequence; performing attention calculation on the activation value according to the forward propagation output result and the key value corresponding to the current text segment to obtain a gradient value; unloading the gradient value to a central processing unit; under the condition that GPU resources are limited, the execution efficiency of a text training task of a large language model can be improved.

The invention discloses a block sensing mask evolution method for efficient NM sparse training, and relates to the technical field of neural network compression and acceleration. The method comprises the following steps: for an N: M sparse mode, dividing a weight vector into weight blocks with a fixed size of M; a weight index is calculated for each weight block, and then a corresponding binary mask is adaptively obtained according to the weight index; calculating a sparse architecture difference for each group of weight blocks to obtain a block sensing score, and then selecting a high-frequency mask block; the binary mask and the high-frequency mask block are combined to realize joint optimization, a new mask is obtained, the original mask is updated through the new mask, pruning and weight recovery are performed, and the training loss is reduced. According to the block perception mask evolution method for efficient NM sparse training, sparse forward and backward propagation is executed in the whole training process, and the training cost is greatly reduced.

The invention belongs to the field of computer vision and cross-modal retrieval, and particularly relates to an image-text pedestrian retrieval method based on image block replacement and cross-modal identity alignment, which comprises the following steps: acquiring a public data set of image and text description, and constructing a pedestrian re-recognition model PRCIA; inputting the data set into a PRCIA model for training and verification, and performing iterative updating on a training weight file through forward and backward propagation to obtain a trained PRCIA model; constructing a reasoning stage model, reserving double encoders and fusing global features in a reasoning stage, and ensuring the calculation efficiency; and inputting the test set into the reasoning stage model to obtain a detection result, thereby realizing text-based pedestrian detection. According to the method, fine-grained association between the image blocks and the text phrases is established through the PR module, cross-modal identity feature expression is enhanced through the CIA module, the accuracy and robustness of text-to-image pedestrian retrieval are remarkably improved, and the requirements of actual scenes are more easily met.

The embodiment of the invention provides a formation imaging system, a formation imaging method, formation imaging equipment and a medium. The system performs formation imaging based on a reverse time migration formation imaging algorithm. The system comprises at least one group of nodes and a CXL2.0 memory pool communicating with the at least one group of nodes, the at least one group of nodes are used for inputting calculation process data into the CXL2.0 memory pool in a forward propagation simulation stage of the reverse time migration stratum imaging algorithm; and reading calculation process data from the CXL2.0 memory pool in a backward propagation simulation stage of the reverse time migration stratigraphic imaging algorithm, and the CXL2.0 memory pool is used for dynamically allocating memory to at least one group of nodes. According to the embodiment of the invention, the dynamic requirement of a reverse time migration (RTM) stratigraphic imaging algorithm in the cluster for the memory can be met, and the effective management of the public memory pool in the cluster is realized, so that the consumption of hardware resources is reduced.

The present invention discloses a calculation method, a calculation system and a storage medium for an attention mechanism. The calculation method includes a forward propagation stage and a backward propagation stage; in the forward propagation stage, operators for QKV mapping, updating KV cache and rotary position encoding are fused, and the internal calculation process of the fused operator is adjusted to reduce the time overhead of memory access; in the backward propagation stage, the calculation order and the memory access order are adjusted to improve the efficiency of the backward propagation process. By optimizing the forward propagation and backward propagation processes of attention calculation, and respectively adopting the strategies of operator fusion and calculation process rearrangement, this method reduces the number of memory accesses and the amount of memory access in the calculation, and improves the training and inference performance of the large language model by accelerating the attention calculation process in the large language model.

The invention provides an integrated circuit testability and test point insertion method, device, equipment and medium, and is applied to the field of integrated circuit testability design, and the method comprises the steps: firstly, judging violation setting types (such as one or more combinations of '-from', '-through' and '-to') existing on a path, and then, according to different time sequence violation settings, carrying out the test point insertion according to different time sequence violation settings. Different test point insertion strategies are adopted: for '-from' and '-through' setting, a test point is inserted behind the last non-'-to' path, so that the test point is utilized to prevent X-state backward propagation; and for the '-to' setting, related circuit nodes are masked (hidden or shielded) so as to control the X state not to influence the subsequent test. After the test points are inserted, the derivation difficulty, the operation time and the vector number of automatic test vector generation can be remarkably reduced, meanwhile, the test coverage rate is improved, and the method has wide applicability and operability.

The invention relates to the technical field of water body recognition, in particular to a black and odorous water body recognition method and system based on artificial intelligence, and the method comprises the following steps: collecting a water body image, carrying out the pixel-level analysis of the water body image, calculating the spectral response of each pixel, analyzing the color distribution, and comparing the spectral characteristics with the color standard. And screening the image areas matched with the black and odorous characteristics to generate spectral characteristic data. Through deep analysis of spectral response and color distribution of each pixel, pollution characteristics in a water body can be identified on a microscopic level, neural network weight adjustment and image processing can be optimized, images of various scales and angles can be effectively processed, generalization ability and environmental adaptability of the model are enhanced, and the method is suitable for being applied to the field of water quality detection. Through detailed forward and backward propagation optimization, the performance of the network in distinguishing water body images with different pollution degrees is remarkably improved, an accurate pollution grade evaluation result can be output, and data support is provided for formulating more targeted water quality management and pollution abatement measures.

The invention provides a high-performance parallel implementation method based on Transform transplantation of a SW processor, and relates to the technical field of processor calculation parallel implementation, slave core calculation is performed by using a SW Athread programming method, each independent function of forward propagation and backward propagation of a ViT algorithm is rewritten into a unified function, a calculation intermediate variable in the forward function is simplified, and the calculation efficiency is improved. Transplanting operation of the ViT algorithm is realized; according to the method, an intermediate variable in a forward function calculation process is simplified, a ViT algorithm based on a Transform architecture is successfully transplanted and operated on a domestic SW processor, unnecessary redundant calculation is removed through an optimization algorithm, and the operation efficiency of a ViT model on a Shenwei super computer is remarkably improved.

The invention belongs to the field of federated learning technology application, and particularly relates to a personalized federated learning method based on a multi-granularity calculation unit, which comprises the following steps: taking a global client as a calculation unit at a coarse granularity level, calculating the membership degree of each cluster by each client, and then voting to select the cluster to which the client belongs, initially aggregating the clients with similar data distribution conditions together; a client cluster is used as a computing unit on a medium granularity level, and a cluster model and a client model are dynamically integrated; a single client is used as a computing unit on a fine-grained level, and implementation of personalized federated learning is accelerated by adjusting computing participation degrees of different layers of a neural network in a federated process, adopting a sparse weight activation training mode and executing sparse convolution in forward and backward propagation. According to the method provided by the invention, from the perspective of a multi-granularity calculation unit, under a data heterogeneous scene, the performance of a client model is effectively improved, and the calculation overhead of federal training is reduced.

The invention discloses a camouflage image generation method for an SAR (Synthetic Aperture Radar) image. The camouflage image generation method comprises the following steps of describing a process of generating the SAR image by a 3D (Three-Dimensional) model as follows: initializing C by using # imgabs0 # and # imgabs1 #; starting iteration; updating a mask vector and # imgabs2 # to carry out post-processing on the C to obtain a two-dimensional SAR image; inputting the image into an automatic target identification classification network to obtain a prediction label; calculating a loss function; and updating # imgabs3 # by using a gradient backward propagation algorithm, and after iteration is finished, generating a final disguised image according to the mapping model. The invention provides a camouflage image generation method based on an SAR imaging mechanism for the first time, and a camouflage image conforming to SAR imaging characteristics can be generated; a RaySAR-based physical countermeasure attack pipeline is realized, and the effectiveness of disguise is proved by testing.

The invention discloses a distributed AI training-oriented RDMA transmission mode intelligent selection method, and the method comprises the steps: obtaining multi-dimensional data features in a deep learning training process, the multi-dimensional data features comprising a tensor structure feature, a data access mode feature, a data timeliness feature, a data dependency feature and a data change rate feature; identifying a current training stage, wherein the training stage comprises a forward propagation stage, a back propagation stage and a parameter updating stage; based on the multi-dimensional data features and the training stage, an optimal RDMA transmission mode matched with the current training scene is selected through an adaptive decision matrix, and the optimal RDMA transmission mode comprises one or more combinations of RDMA Write operation, RDMA Read operation, Send / Receive operation and Atomic operation; and switching the RDMA transmission mode in real time according to the dynamic change of a network environment, ensuring that a training data flow is not interrupted in a switching process by adopting a progressive flow migration strategy, and optimizing a subsequent mode selection decision through a historical performance learning mechanism.

The invention discloses a bidirectional adaptive video super-resolution method based on frame difficulty index, and belongs to the field of computer vision. The invention provides a frame-level dynamic reconstruction method for solving the problems that simple frame calculation is redundant and difficult frame reconstruction is insufficient due to the fact that an existing model adopts a fixed calculation strategy for video frames with different difficulties. According to the method, a motion detail decoupling propagation network is constructed, motion information is efficiently transmitted by utilizing a shallow forward propagation branch, and a deep backward propagation branch focuses on recovering texture details so as to decouple a time sequence propagation task; meanwhile, a frame reconstruction difficulty evaluation network is introduced to generate a global difficulty index, so that the receptive field weight of the adaptive time sequence fusion network and the refining depth of the dynamic refining network are regulated and controlled. According to the method, through explicit modeling frame-level reconstruction difficulty, adaptive matching of the model capacity and the video frame feature complexity is realized, and the video reconstruction performance is remarkably improved under limited computing power.

The invention discloses a rural domestic sewage treatment system and method for controlling an A2O reaction process based on energy detection. The system is provided with a grating well, a regulating tank, an anaerobic-anoxic zone, an aerobic zone and a sedimentation tank, and is equipped with a monitoring module for monitoring the temperature and water level of sewage in each section so as to obtain volume information. The control method adopts a long short-term memory (LSTM) neural network model to perform reaction process prediction, and specifically comprises the following steps: firstly, performing normalization processing on monitored temperature and water level data, and constructing an input feature vector; defining an LSTM model structure and dividing a data set; model training is completed through forward propagation calculation, loss function calculation and back propagation updating parameters; and finally, predicting anaerobic, anoxic and aerobic zone reaction processes by using the trained model, and adjusting parameters such as an internal reflux ratio according to a prediction result. The method can accurately grasp the reaction process, is suitable for the characteristic of large fluctuation of the quality and quantity of rural sewage, improves the sewage treatment effect, reduces the operation and maintenance cost, and achieves data-driven intelligent decision making.

Method comprising: monitoring whether a MTLF receives a first state of an environment on which a RL training is to be performed; performing a ML model forward propagation on a first model of the environment having the first state for each of plural actions to obtain a respective expected reward for each of the plural actions: informing a service consumer on the plural actions and their respective expected reward; supervising whether the MTLF receives a RL training result information after the informing the service consumer on the plural actions, wherein the RL training result information comprises an indication of one of the plural actions, a second state of the environment, and a reward feedback; conducting a ML model backward propagation on the first model of the environment having the second state for the one of the plural actions using the reward feedback to obtain a second model of the environment.

The invention discloses a data processing method and device, electronic equipment and a storage medium. The data processing method comprises the following steps: acquiring a computational graph; determining at least one core operator based on the computational graph; determining arrangement modes corresponding to the plurality of operators through at least one round of iterative operation; wherein in each round of iterative operation, one or more core operators are selected from at least one core operator to serve as initial operators; determining an arrangement mode corresponding to each initial operator according to the read-write overhead of each initial operator in a plurality of preset arrangement modes; the initial operator is taken as a starting point, forward and backward propagation is carried out according to the dependency relationship in combination with the arrangement mode corresponding to the initial operator, the arrangement mode corresponding to each operator is obtained in sequence, and in the process of obtaining the arrangement mode corresponding to each operator in sequence, the arrangement mode corresponding to the current operator is obtained in combination with the arrangement mode corresponding to the adjacent operator; the adjacent operators are operators which are adjacent to the current operator according to the dependency relationship and have determined arrangement modes.

Method for training an artificial neural network (ANN), wherein the method exhibits: Storing weight values ​​in an array of resistive processing units (12), RPU, wherein the array of RPU units (12) represents a weight matrix (10) W of the ANN with m rows and n columns, by storing the weight values ​​of the weight matrix (10) W as resistive values ​​of the RPU units in the array; Define the weighting matrix (10) W such that it has an output dimension that is smaller than the input dimension, so that the weighting matrix (10) W has a rectangular shape; and Transforming (1010) the weighting matrix (10) W from a rectangular shape to a more square shape by repeating or concatenating the rectangular shape of the weighting matrix (10) W to increase the signal strength of a reverse pass signal by: Copying (1020) an input of repeated weighting elements during a forward cycle pass; Summing (1030) output calculations from the repeated weighting elements, resulting in one output per line; and Update (1040) each of the repeated weight elements according to a backward propagated error or alternatively update (1040) only one of the repeated weight elements by setting all forward values ​​except one to zero during an update pass.

This invention relates to a method and system for detecting atmospheric turbulence intensity based on a photon diffraction neural network. The method includes: establishing and training a photon diffraction neural network, which includes a forward propagation model and a backward propagation model, wherein the backward propagation model is an error backpropagation based on a loss function, used to train the photon diffraction neural network; acquiring light beams under different atmospheric intensities; inputting the light beams into the trained photon diffraction neural network; detecting the output light field intensity of the photon diffraction neural network based on a light intensity detector; and completing the atmospheric turbulence intensity detection according to the correspondence between the light field intensity and the atmospheric turbulence intensity, wherein the stronger the atmospheric turbulence intensity, the weaker the light field intensity. Compared with the prior art, this invention has the advantages of requiring no additional signal processing, fast detection speed, and high detection efficiency.