13 results about "Feed forward neural" patented technology

Aspects of the disclosure include methods for leveraging retrieval augmented generation (RAG) over a graph neural network (GNN) for edge building and the generation of reason-aware graph recommendations. A method can include constructing a graph neural network from an input graph having a plurality of nodes and one or more edges. The graph neural network includes one or more internal layers, each internal layer having one or more node vectors encoding a K-hop neighborhood for a target node of the plurality of nodes. RAG data including non-graph contextual data is retrieved for each of the plurality of nodes and transformed into embeddings using a large language model encoder. The RAG embeddings are encoded into node vectors of the graph neural network. The graph neural network generates a representation for the target node that is transformed by a feed forward neural network tower into an output vector.

A system applies an AI-based model pipeline for joint prediction of repair time and repair actions based on a received fault report related to manufacturing automation machines and equipment. Model pipeline includes a first encoder configured as a transformer neural network trained to transform fault report text to a latent fault report representation that groups semantically similar fault texts. Another transformer encoder is trained to transform the latent fault report representation to a latent repair report representation. Repair time prediction module with a feed forward neural network is trained to generate repair time predictions from the latent repair report representation against actual repair times. Repair action prediction module configured as a transformer decoder neural network is trained to generate repair action predictions from the latent repair report representation against actual repair actions. End-to-end training of the pipeline is achieved by minimizing a combined loss function of the joint prediction.

Embodiments of the present disclosure provide a data processing method and device, electronic equipment and storage medium. The method comprises: acquiring an image, determining a to-be-processed feature corresponding to the image; inputting the to-be-processed feature into an autoregressive sequence generation model to obtain a target feature of the image; wherein the autoregressive sequence generation model comprises a lightweight self-attention subnetwork and a feedforward neural subnetwork; and performing analysis and processing on the image based on the target feature. The technical solution of the embodiments of the present disclosure reduces the requirement of the model on computing power, and even if the model is deployed on a mobile terminal, good data processing effect can be achieved. At the same time, the calculation amount in the model running process is reduced, and real-time processing of related data is facilitated.

The application discloses an image classification system and method based on a lightweight LA Transformer network, comprising a feature extraction module, an LA Transformer main network and an image classification module. The feature extraction module is responsible for local feature extraction of an input RGB image, realizes down-sampling of the input image, and outputs the image as an input of the LA Transformer main network. The LA Transformer main network is responsible for further feature extraction of the image. A local self-attention subnetwork is used to model spatial correlation information between feature map patches and patches by using a local self-attention mechanism, and an attention feedforward neural subnetwork is used to model correlation information between feature map channels and channels. The image classification processing module is responsible for generating a probability for each image classification, and completes classification of the input RGB image. The application can achieve higher image classification accuracy while having lower parameter quantity and calculation amount.

The present invention discloses a millimeter wave radar sparse point cloud reconstruction method based on a multi-step neural network structure, comprising step S1: obtaining first information including speed, angle and distance through a millimeter wave radar and performing Fourier transform processing on the obtained speed and distance, so as to use the generated speed and distance spectrum as the first target feature. The present invention discloses a millimeter wave radar sparse point cloud reconstruction method based on a multi-step neural network structure, which directly naturalizes radar sparse point cloud information to predict high-resolution point cloud information through a feed forward neural network, and provides a new algorithm for the millimeter wave radar sparse point cloud problem.

The invention discloses a crop pest detection and early warning system based on a fruit fly perception neural network in the field of pest monitoring and early warning, the crop pest detection and early warning system comprises a pre-synaptic nervous system network and a post-synaptic nervous system network, the pre-synaptic nervous system network is used for processing video information, and the post-synaptic nervous system network is used for constructing an early warning monitoring model. The visual field information is processed based on biological inspiration of a fruit fly visual system network. The APDNNM visual neural network model is used for solving the technical problem of pest detection in intelligent video monitoring. According to the model, spike neuronal cells are constructed and synthesized on the basis of a response characteristic mechanism of feedforward neuronal cells and LPTC neurons of a fruit fly visual nervous system to sense pest information existing in crops, and early warning detection is achieved.

The invention discloses a model loading and unloading method and electronic equipment, and relates to the technical field of server resource management.The method comprises the steps that feedforward neural sub-networks are preloaded to preset hardware layer by layer according to a preset sequence and historical operation process data; performing first calculation on the first resource use information, the second resource use information and the memory use information according to a preset time interval to obtain a first resource use function value; performing second calculation on the second resource use information and the memory use information according to a preset time interval to obtain a second resource use function value; when it is detected that the second resource use function value is smaller than a second predefined resource use function value threshold value lower bound, the feedforward neural sub-network in the running process is unloaded from the first processor to the second processor. According to the invention, the problem of low model feedback efficiency caused by unbalanced distribution of each sub-network in each hardware of the server in the model in the related technology is solved.

The present invention is a CVT operating state evaluation method and system based on an improved decoding layer model, and relates to the technical field of online monitoring of electric power metering, wherein the method comprises acquiring CVT data for hysteresis operation; constructing a CVT operating state evaluation model, comprising an embedding layer, an improved transformer decoding layer, a CloFormer layer and a distribution estimation layer; the improved transformer decoding layer comprises a first normalization layer, an attention layer, a second normalization layer and a feedforward neural layer; an embedded variable is generated through the embedding layer, and is input into the improved transformer decoding layer after normalization; a CloFormer layer is used for feature extraction; the distribution estimation layer is a Gaussian distribution model, and an optimal parameter group is determined by using an EM algorithm and a BIC criterion, and a model training is performed to obtain a final CVT operating state evaluation model; and real-time CVT data is input into the final model to obtain the CVT operating state.

The invention discloses a CVT running state evaluation method and system based on an improved decoding layer model, and relates to the technical field of electric power measurement online monitoring, and the method comprises the steps: obtaining CVT data, and carrying out the hysteresis operation; constructing a CVT operation state evaluation model, wherein the CVT operation state evaluation model comprises an embedding layer, an improved transformer decoding layer, a CloFormer layer and a distribution estimation layer; the improved transform decoding layer comprises a first normalization layer, an attention layer, a second normalization layer and a feedforward neural layer; generating an embedded variable through an embedded layer, normalizing the embedded variable, and inputting the normalized embedded variable into an improved transform decoding layer; carrying out feature extraction by utilizing a CloFormer layer; the distribution estimation layer is a Gaussian distribution model, an optimal parameter group is determined by using an EM algorithm and a BIC criterion, and model training is carried out to obtain a final CVT operation state evaluation model; and inputting real-time CVT data into the final model to obtain the running state of the CVT.

Apparatus and methods for a pattern identification transformer neural network is provided. The pattern identification transformer neural network may be able to learn from relatively small numbers of data elements. The pattern identification transformer neural network may function in similar method to the way humans transform data points. As such, the pattern identification transformer neural network may be able to learn patterns from a small number of examples and determine what attributes are helpful from a single experience. The pattern identification transformer neural networks may include a multi-head attention module, a normalize module and a feed forward neural network. The multi-head attention module may receive vectors that correspond to experiences. The normalize module may normalize the received vectors. The feed forward neural network may incorporate the received vectors into the neural network.

This application discloses a model loading / unloading method and electronic device, relating to the field of server resource management technology. The method includes: preloading each feedforward neural subnetwork onto preset hardware in a preset order based on historical operation data; performing a first calculation on first resource usage information, second resource usage information, and memory usage information at preset time intervals to obtain a first resource usage function value; performing a second calculation on second resource usage information and memory usage information at preset time intervals to obtain a second resource usage function value; and unloading the feedforward neural subnetwork from a first processor to a second processor when the second resource usage function value is detected to be less than a second predefined lower bound of a resource usage function value threshold. This invention solves the problem of low model feedback efficiency caused by the uneven distribution of subnetworks in the model across different hardware components of the server in related technologies.

The invention relates to a space-time sequence prediction method, and belongs to the field of artificial intelligence time sequence prediction. The method comprises the following steps: S1, processing a node time sequence by adopting a sliding window sampling method; s2, performing random masking on the samples; s3, capturing a hidden representation of the long-time sub-sequence sample based on a multi-head attention mechanism; s4, obtaining an incidence matrix of the hypergraph based on the nodes and the hyperedge representation matrix; s5, updating the node information; s6, updating the hyperedge information; s7, defining hyperedge and node constraint functions; s8, inputting the short-time sub-sequence samples into a gating recursion unit to obtain hidden representation; and S9, clustering the nodes based on a K-means algorithm, and inputting the hidden representations of the long-time and short-time sub-sequence samples into a feedforward neural layer to obtain a prediction result of the space-time sequence.

The application discloses a model deployment method and device, and the method comprises the following steps: acquiring a pre-trained visual convolution model, setting an accuracy target and search parameters for pruning the visual convolution model, wherein the search parameters at least comprise an initial pruning ratio and a search interval length; searching a first target pruning ratio of an attention mechanism module and a second target pruning ratio of a feedforward neural module respectively; optimizing the visual convolution model according to the first target pruning ratio and the second target pruning ratio to obtain a target visual convolution model; and deploying the target visual convolution model on an edge computing device of a substation. In this way, after the visual convolution model is pruned, the model size is in a minimum state, so that the requirement for the running environment is reduced. The visual convolution model is deployed on the edge computing device, real-time reception of monitoring data is realized, so that real-time detection and warning can be achieved, and the safety of the power system is improved.