209 results about "Backward propagation" patented technology

The invention discloses a classification model training method based on big data distributed computing, and relates to the technical field of big data processing and machine learning, and the method comprises the steps: S1, obtaining communication delay data between distributed computing nodes, and analyzing delay characteristics to generate regulation and control parameters, the method comprises the steps of S1, dynamically determining the batch size of each training batch based on regulation and control parameters, and ensuring the optimization of the synchronization efficiency, S3, distributing the adjusted batch size to each node, and executing the forward and reverse propagation calculation process of the model, and S4, collecting the calculation result of each node to perform parameter aggregation updating, and completing the iterative training of the classification model. According to the classification model training method based on big data distributed computing, the batch size is regulated and controlled according to the communication delay between the nodes so as to solve the problem that the distributed synchronization efficiency is low.

The invention relates to the field of federated learning security, and provides a privacy protection longitudinal federated learning method and system based on function encryption. According to the method, the problems that in a longitudinal federated learning scene, when privacy protection is achieved based on the cryptography technology, the calculation time is too long, the communication overhead is too large, and the model accuracy and the training overhead are difficult to balance are solved. The main scheme comprises system initialization, client forward propagation, server forward propagation, server backward propagation and client backward propagation. The method is used for privacy protection longitudinal federated learning, gradient leakage attacks and reverse derivation attacks based on model parameters are effectively resisted through a decentralized multi-client function encryption algorithm and a longitudinal federated learning framework, the calculation efficiency is improved, meanwhile, the model precision is guaranteed, the communication overhead is reduced, and the privacy of client sample data is enhanced.

The invention discloses a power load prediction model establishment method based on a convolutional bidirectional long-short-term memory network and an attention mechanism, and the method comprises the steps: carrying out the full feature extraction of original complex power load data through a one-dimensional convolutional network; the factors influencing the load and the characteristics of trend, periodicity, randomness and the like of load change can be more fully mastered by considering the multivariate characteristics; compared with the traditional LSTM, the bidirectional structure of the BiLSTM can more comprehensively capture the bidirectional dependency relationship in the load sequence data through forward propagation and backward propagation, and the understanding of the model on the data is improved. Besides, an attention mechanism widely applied in the field of natural language processing is applied to research on multi-element short-term power load prediction, key information in a load sequence can be focused by introducing the attention mechanism, and the prediction precision is further improved; experiments are carried out on a real multivariate load data set, and the precision of multivariate short-term load prediction can be effectively improved by adding the one-dimensional convolutional network and the attention mechanism.

The invention relates to a quantitative perception training method and device of a neural network model, electronic equipment and a storage medium. The method comprises the steps of obtaining a to-be-trained first neural network model; an operator pair and a non-module operator in the first neural network model are identified, the non-module operator is an operation which is not realized based on a module class, and the operator pair comprises a convolution operator and a batch normalization operator which are connected; the non-module operators are packaged into module operators, the operator pairs are packaged into new convolution operators, a second neural network model is obtained, the new convolution operators run the fusion process of the convolution operators and batch normalization operators during forward propagation, and the parameters of the convolution operators and the batch normalization operators are updated during back propagation; and performing quantitative perception training on the second neural network model. By adopting the method, the reasoning precision of the quantitative model can be guaranteed, and the deployment efficiency of the quantitative model is improved.

The invention relates to the technical field of deep learning, and discloses a fine-grained pipeline scheduling method and device based on cluster node memory difference perception. The method comprises the following steps: estimating the ratio of the peak memory occupancy to the memory capacity of each GPU node, comparing the ratio with a preset threshold value, and selecting a re-calculation strategy and a back propagation segmentation strategy: by taking minimization of the end-to-end training time of a model as a target, scheduling and modeling micro-batch data as a flow shop problem, analyzing the dependency constraint of each flow line stage, and calculating the flow shop problem; comprising forward-back propagation sequence dependence, stage dependence, operation dependence and memory limitation, and generating a micro-batch operation sequence scheduling scheme meeting the memory capacity limitation; according to the scheduling scheme, an execution sequence queue is created by executing sorting, and the execution time sequence of calculation blocks, communication blocks and re-calculation operation is dynamically coordinated. According to the method, the GPU equipment utilization rate can be remarkably improved, and the end-to-end training completion time of the model is shortened.

The invention belongs to the technical field of target detection, and particularly relates to a teacher tensor knowledge distillation-driven anti-violation target detection method in an electric power safety supervision scene, and the method comprises the following steps: inputting image data to a student model and a pre-trained teacher model; performing forward propagation on the student model to obtain a student prediction tensor; associating differences between the real tags based on the student prediction tensor and the input image data; acquiring an original teacher output tensor by using the same input image data by calling a bottom layer forward propagation method of a teacher model; calculating knowledge distillation loss based on the difference between the student prediction tensor and the original teacher output tensor; the standard detection loss and the knowledge distillation loss are combined to form total loss; and performing back propagation updating on parameters of the student model based on the total loss so as to complete model training and real-time target detection and detection method updating optimization. According to the invention, the performance of the student model can be significantly improved, and the high efficiency of the model is maintained.

Embodiments of the invention disclose a slow node detection method and apparatus, a device, a storage medium and a program product. The method comprises the steps of obtaining respective first duration data of a plurality of processing nodes participating in distributed model training; the first duration data represents the calculation time consumption of the corresponding processing node in the current iteration; based on a parallel strategy trained by the distributed model, grouping the first duration data to obtain a first duration set of multiple groups; the parallel strategy represents an association relationship of the plurality of processing nodes in a forward and backward propagation process and training tasks undertaken by the plurality of processing nodes; and based on the respective probability density function and a first threshold value of the plurality of groups, respectively analyzing and processing the first duration sets of the plurality of groups to obtain slow nodes in the plurality of processing nodes. Thus, the slow nodes in distributed model training can be intelligently and accurately detected, the efficiency and accuracy of slow node detection are improved, and the method has high compatibility.

The invention relates to a model training task execution method, a graphics processing unit and a model training system.The model training task execution method comprises the steps that when attention calculation of model forward propagation is executed on a current text segment, the current text segment is subjected to attention calculation; sequentially loading key values corresponding to all the preamble segments from the central processing unit to carry out attention calculation on the activation values; meanwhile, the key value pair activation value corresponding to the current text segment is unloaded to the central processing unit; when the attention calculation of model back propagation is executed on the current text segment, loading a forward propagation output result and a key value pair activation value corresponding to the current text segment from a central processing unit, and loading the attention activation value corresponding to each preorder text segment in sequence; performing attention calculation on the activation value according to the forward propagation output result and the key value corresponding to the current text segment to obtain a gradient value; unloading the gradient value to a central processing unit; under the condition that GPU resources are limited, the execution efficiency of a text training task of a large language model can be improved.

The invention belongs to the field of computer vision and cross-modal retrieval, and particularly relates to an image-text pedestrian retrieval method based on image block replacement and cross-modal identity alignment, which comprises the following steps: acquiring a public data set of image and text description, and constructing a pedestrian re-recognition model PRCIA; inputting the data set into a PRCIA model for training and verification, and performing iterative updating on a training weight file through forward and backward propagation to obtain a trained PRCIA model; constructing a reasoning stage model, reserving double encoders and fusing global features in a reasoning stage, and ensuring the calculation efficiency; and inputting the test set into the reasoning stage model to obtain a detection result, thereby realizing text-based pedestrian detection. According to the method, fine-grained association between the image blocks and the text phrases is established through the PR module, cross-modal identity feature expression is enhanced through the CIA module, the accuracy and robustness of text-to-image pedestrian retrieval are remarkably improved, and the requirements of actual scenes are more easily met.

The embodiment of the invention provides a formation imaging system, a formation imaging method, formation imaging equipment and a medium. The system performs formation imaging based on a reverse time migration formation imaging algorithm. The system comprises at least one group of nodes and a CXL2.0 memory pool communicating with the at least one group of nodes, the at least one group of nodes are used for inputting calculation process data into the CXL2.0 memory pool in a forward propagation simulation stage of the reverse time migration stratum imaging algorithm; and reading calculation process data from the CXL2.0 memory pool in a backward propagation simulation stage of the reverse time migration stratigraphic imaging algorithm, and the CXL2.0 memory pool is used for dynamically allocating memory to at least one group of nodes. According to the embodiment of the invention, the dynamic requirement of a reverse time migration (RTM) stratigraphic imaging algorithm in the cluster for the memory can be met, and the effective management of the public memory pool in the cluster is realized, so that the consumption of hardware resources is reduced.

The present invention discloses a calculation method, a calculation system and a storage medium for an attention mechanism. The calculation method includes a forward propagation stage and a backward propagation stage; in the forward propagation stage, operators for QKV mapping, updating KV cache and rotary position encoding are fused, and the internal calculation process of the fused operator is adjusted to reduce the time overhead of memory access; in the backward propagation stage, the calculation order and the memory access order are adjusted to improve the efficiency of the backward propagation process. By optimizing the forward propagation and backward propagation processes of attention calculation, and respectively adopting the strategies of operator fusion and calculation process rearrangement, this method reduces the number of memory accesses and the amount of memory access in the calculation, and improves the training and inference performance of the large language model by accelerating the attention calculation process in the large language model.

The invention relates to the technical field of water body recognition, in particular to a black and odorous water body recognition method and system based on artificial intelligence, and the method comprises the following steps: collecting a water body image, carrying out the pixel-level analysis of the water body image, calculating the spectral response of each pixel, analyzing the color distribution, and comparing the spectral characteristics with the color standard. And screening the image areas matched with the black and odorous characteristics to generate spectral characteristic data. Through deep analysis of spectral response and color distribution of each pixel, pollution characteristics in a water body can be identified on a microscopic level, neural network weight adjustment and image processing can be optimized, images of various scales and angles can be effectively processed, generalization ability and environmental adaptability of the model are enhanced, and the method is suitable for being applied to the field of water quality detection. Through detailed forward and backward propagation optimization, the performance of the network in distinguishing water body images with different pollution degrees is remarkably improved, an accurate pollution grade evaluation result can be output, and data support is provided for formulating more targeted water quality management and pollution abatement measures.

The invention belongs to the field of federated learning technology application, and particularly relates to a personalized federated learning method based on a multi-granularity calculation unit, which comprises the following steps: taking a global client as a calculation unit at a coarse granularity level, calculating the membership degree of each cluster by each client, and then voting to select the cluster to which the client belongs, initially aggregating the clients with similar data distribution conditions together; a client cluster is used as a computing unit on a medium granularity level, and a cluster model and a client model are dynamically integrated; a single client is used as a computing unit on a fine-grained level, and implementation of personalized federated learning is accelerated by adjusting computing participation degrees of different layers of a neural network in a federated process, adopting a sparse weight activation training mode and executing sparse convolution in forward and backward propagation. According to the method provided by the invention, from the perspective of a multi-granularity calculation unit, under a data heterogeneous scene, the performance of a client model is effectively improved, and the calculation overhead of federal training is reduced.

The invention discloses a camouflage image generation method for an SAR (Synthetic Aperture Radar) image. The camouflage image generation method comprises the following steps of describing a process of generating the SAR image by a 3D (Three-Dimensional) model as follows: initializing C by using # imgabs0 # and # imgabs1 #; starting iteration; updating a mask vector and # imgabs2 # to carry out post-processing on the C to obtain a two-dimensional SAR image; inputting the image into an automatic target identification classification network to obtain a prediction label; calculating a loss function; and updating # imgabs3 # by using a gradient backward propagation algorithm, and after iteration is finished, generating a final disguised image according to the mapping model. The invention provides a camouflage image generation method based on an SAR imaging mechanism for the first time, and a camouflage image conforming to SAR imaging characteristics can be generated; a RaySAR-based physical countermeasure attack pipeline is realized, and the effectiveness of disguise is proved by testing.

The invention discloses a distributed AI training-oriented RDMA transmission mode intelligent selection method, and the method comprises the steps: obtaining multi-dimensional data features in a deep learning training process, the multi-dimensional data features comprising a tensor structure feature, a data access mode feature, a data timeliness feature, a data dependency feature and a data change rate feature; identifying a current training stage, wherein the training stage comprises a forward propagation stage, a back propagation stage and a parameter updating stage; based on the multi-dimensional data features and the training stage, an optimal RDMA transmission mode matched with the current training scene is selected through an adaptive decision matrix, and the optimal RDMA transmission mode comprises one or more combinations of RDMA Write operation, RDMA Read operation, Send / Receive operation and Atomic operation; and switching the RDMA transmission mode in real time according to the dynamic change of a network environment, ensuring that a training data flow is not interrupted in a switching process by adopting a progressive flow migration strategy, and optimizing a subsequent mode selection decision through a historical performance learning mechanism.

The invention discloses a bidirectional adaptive video super-resolution method based on frame difficulty index, and belongs to the field of computer vision. The invention provides a frame-level dynamic reconstruction method for solving the problems that simple frame calculation is redundant and difficult frame reconstruction is insufficient due to the fact that an existing model adopts a fixed calculation strategy for video frames with different difficulties. According to the method, a motion detail decoupling propagation network is constructed, motion information is efficiently transmitted by utilizing a shallow forward propagation branch, and a deep backward propagation branch focuses on recovering texture details so as to decouple a time sequence propagation task; meanwhile, a frame reconstruction difficulty evaluation network is introduced to generate a global difficulty index, so that the receptive field weight of the adaptive time sequence fusion network and the refining depth of the dynamic refining network are regulated and controlled. According to the method, through explicit modeling frame-level reconstruction difficulty, adaptive matching of the model capacity and the video frame feature complexity is realized, and the video reconstruction performance is remarkably improved under limited computing power.

The invention discloses a rural domestic sewage treatment system and method for controlling an A2O reaction process based on energy detection. The system is provided with a grating well, a regulating tank, an anaerobic-anoxic zone, an aerobic zone and a sedimentation tank, and is equipped with a monitoring module for monitoring the temperature and water level of sewage in each section so as to obtain volume information. The control method adopts a long short-term memory (LSTM) neural network model to perform reaction process prediction, and specifically comprises the following steps: firstly, performing normalization processing on monitored temperature and water level data, and constructing an input feature vector; defining an LSTM model structure and dividing a data set; model training is completed through forward propagation calculation, loss function calculation and back propagation updating parameters; and finally, predicting anaerobic, anoxic and aerobic zone reaction processes by using the trained model, and adjusting parameters such as an internal reflux ratio according to a prediction result. The method can accurately grasp the reaction process, is suitable for the characteristic of large fluctuation of the quality and quantity of rural sewage, improves the sewage treatment effect, reduces the operation and maintenance cost, and achieves data-driven intelligent decision making.

Method comprising: monitoring whether a MTLF receives a first state of an environment on which a RL training is to be performed; performing a ML model forward propagation on a first model of the environment having the first state for each of plural actions to obtain a respective expected reward for each of the plural actions: informing a service consumer on the plural actions and their respective expected reward; supervising whether the MTLF receives a RL training result information after the informing the service consumer on the plural actions, wherein the RL training result information comprises an indication of one of the plural actions, a second state of the environment, and a reward feedback; conducting a ML model backward propagation on the first model of the environment having the second state for the one of the plural actions using the reward feedback to obtain a second model of the environment.

The invention discloses a data processing method and device, electronic equipment and a storage medium. The data processing method comprises the following steps: acquiring a computational graph; determining at least one core operator based on the computational graph; determining arrangement modes corresponding to the plurality of operators through at least one round of iterative operation; wherein in each round of iterative operation, one or more core operators are selected from at least one core operator to serve as initial operators; determining an arrangement mode corresponding to each initial operator according to the read-write overhead of each initial operator in a plurality of preset arrangement modes; the initial operator is taken as a starting point, forward and backward propagation is carried out according to the dependency relationship in combination with the arrangement mode corresponding to the initial operator, the arrangement mode corresponding to each operator is obtained in sequence, and in the process of obtaining the arrangement mode corresponding to each operator in sequence, the arrangement mode corresponding to the current operator is obtained in combination with the arrangement mode corresponding to the adjacent operator; the adjacent operators are operators which are adjacent to the current operator according to the dependency relationship and have determined arrangement modes.

Method for training an artificial neural network (ANN), wherein the method exhibits: Storing weight values ​​in an array of resistive processing units (12), RPU, wherein the array of RPU units (12) represents a weight matrix (10) W of the ANN with m rows and n columns, by storing the weight values ​​of the weight matrix (10) W as resistive values ​​of the RPU units in the array; Define the weighting matrix (10) W such that it has an output dimension that is smaller than the input dimension, so that the weighting matrix (10) W has a rectangular shape; and Transforming (1010) the weighting matrix (10) W from a rectangular shape to a more square shape by repeating or concatenating the rectangular shape of the weighting matrix (10) W to increase the signal strength of a reverse pass signal by: Copying (1020) an input of repeated weighting elements during a forward cycle pass; Summing (1030) output calculations from the repeated weighting elements, resulting in one output per line; and Update (1040) each of the repeated weight elements according to a backward propagated error or alternatively update (1040) only one of the repeated weight elements by setting all forward values ​​except one to zero during an update pass.

This invention relates to a method and system for detecting atmospheric turbulence intensity based on a photon diffraction neural network. The method includes: establishing and training a photon diffraction neural network, which includes a forward propagation model and a backward propagation model, wherein the backward propagation model is an error backpropagation based on a loss function, used to train the photon diffraction neural network; acquiring light beams under different atmospheric intensities; inputting the light beams into the trained photon diffraction neural network; detecting the output light field intensity of the photon diffraction neural network based on a light intensity detector; and completing the atmospheric turbulence intensity detection according to the correspondence between the light field intensity and the atmospheric turbulence intensity, wherein the stronger the atmospheric turbulence intensity, the weaker the light field intensity. Compared with the prior art, this invention has the advantages of requiring no additional signal processing, fast detection speed, and high detection efficiency.

This invention relates to a closed-loop data system and method for embodied intelligent robots. The system includes: a data acquisition module for acquiring multimodal operational data of the embodied intelligent robot during task execution and constructing multiple embodied state nodes based on the multimodal operational data; a construction module for constructing event correlation relationships between embodied state nodes based on temporal and physical constraints, forming an embodied state event graph; an anomaly identification module for identifying anomalous embodied state nodes in the embodied state event graph and marking them as failure source nodes; and a propagation analysis module for determining failure propagation paths forward or backward from the failure source nodes based on event correlation relationships. Using the above scheme, the operational data during the task execution of the embodied intelligent robot can be effectively analyzed, anomalous states can be accurately identified, and data-driven closed-loop optimization can be achieved.

Embodiments of the present application provide a model interpretation method and apparatus, a storage medium, and a computer program product. The method comprises: determining an attention weight value of first multimedia data in a forward propagation process in a first model, and determining a gradient value of the first multimedia data in a backward propagation process in the first model, wherein the first multimedia data is any input data input to the first model; using the attention weight value and the gradient value to determine an attention value of the first multimedia data; and using the attention value as interpretability information of the first multimedia data, so as to use the interpretability information to interpret a decision-making process of the first model.

The application discloses an end-side fingerprint representation recognition method based on multi-task learning. First, the fingerprint picture is preprocessed, and then the preprocessed training data is input into a backbone neural network to obtain basic features; then the basic features are input into a minutia extraction network, and texture information maps are generated through multiple layers of convolution and deconvolution. The basic features are input into a multi-layer perceptron to generate topological information and obtain corresponding category information. Finally, the basic features, the texture information and the topological information are updated through a joint loss function and backward propagation in three network modules to achieve the purpose of converting the basic features into fingerprint representation information with the assistance of the texture information and the topological information. The application uses a multi-task learning method to learn fingerprint feature information in multiple scales, effectively improving the fingerprint recognition accuracy. The application fuses fingerprint multi-scale information into a fingerprint representation, making the robustness stronger. The application adopts a lightweight network throughout, saving hardware resource overhead and being more suitable for end-side devices.

Computer systems and computer-implemented methods modify a machine learning network, such as a deep neural network, to introduce judgment to the network. A “combining” node is added to the network, to thereby generate a modified network, where activation of the combining node is based, at least in part, on output from a subject node of the network. The computer system then trains the modified network by, for each training data item in a set of training data, performing forward and back propagation computations through the modified network, where the backward propagation computation through the modified network comprises computing estimated partial derivatives of an error function of an objective for the network, except that the combining node selectively blocks back-propagation of estimated partial derivatives to the subject node, even though activation of the combining node is based on the activation of the subject node.

In some implementations, there is provided receiving a query request including a top k query operator for query plan generation, optimization, and execution; generating a query plan that includes at least one pipeline of a plurality of operators, wherein the at least one pipeline is associated with a directed acyclic graph; detecting in the query plan a first operator in the at least one pipeline that causes an early exit; and in response to the early exit by the first operator during query execution, processing back through the directed acyclic graph to identify at least one preceding operator that should or should not run given the early exit of the first operator.

A compute cluster includes a compute mesh of interconnected compute elements. A clock mesh includes reconvergent elements, and the reconvergent clock mesh distributes a clock signal to the compute elements. The clock mesh has a topology of nodes connected by branches. Repeaters are located on the branches between nodes. As a result, the reconvergent clock mesh is unidirectional. The repeaters limit the clock signal to forward propagation through the clock mesh and prevent backward propagation through the clock mesh. The clock mesh is reconvergent, in that the clock signal may branch out along different paths from one node and then these paths reconverge at a later node.

A specialized integrated circuit for artificial neural networks is integrated with high bandwidth memory. The neural network includes a systolic array of interconnected processing elements, including upstream processing elements and downstream processing elements. Each processing element includes a pair of input / output ports for concurrent forward and backward propagation. The processing elements can be used for convolution, in which case the pair of input / output ports can support fast and efficient scanning of a kernel over activations.

The present invention discloses a dual-path residual structure neural network model and an image target recognition system. The dual-path residual structure neural network model includes at least two cascaded network units in sequence; each network unit includes a plurality of cascaded convolutional neural network modules in sequence; a first residual branch is arranged between the input end and the output end of the network unit; a second residual branch is arranged between two network units; the input and output of the first residual branch are both the image to be recognized; the input and output of the second residual branch are both the output of any one of the convolutional neural network modules in the previous network unit. By setting the first residual branch and the second residual branch, the present invention obtains a dual-path residual structure, which can promote the forward propagation of image features and the backward propagation of gradients, so as to make up for the problem that the prior art cannot fully solve the gradient dispersion problem, and further improve the feature extraction performance and target classification performance of the deep neural network.

The application discloses a medical image class incremental learning method and system based on feature principal direction guided projection, and belongs to the technical field of medical image analysis; the method acquires medical image features through a pre-trained feature extractor; random projection is performed on the features to obtain first-view features; a covariance matrix is calculated based on a feature cache area and is decomposed to extract the first K principal directions, construct a guided projection matrix, and obtain second-view features; the double-view features are spliced to obtain final representation; the classifier weight is directly calculated through a ridge regression closed-form solution without backward propagation training; the application adaptively captures the core structure of the feature space through data-driven principal direction projection, retains the detailed information in combination with random projection, effectively improves the feature separability of fine-grained classes, adopts an analytical classifier updating mechanism, realizes stable class incremental learning without storing historical data, improves the model updating efficiency, and is suitable for efficient continuous learning tasks in various medical image scenes.