7 results about "Feed forward neural" patented technology

Aspects of the disclosure include methods for leveraging retrieval augmented generation (RAG) over a graph neural network (GNN) for edge building and the generation of reason-aware graph recommendations. A method can include constructing a graph neural network from an input graph having a plurality of nodes and one or more edges. The graph neural network includes one or more internal layers, each internal layer having one or more node vectors encoding a K-hop neighborhood for a target node of the plurality of nodes. RAG data including non-graph contextual data is retrieved for each of the plurality of nodes and transformed into embeddings using a large language model encoder. The RAG embeddings are encoded into node vectors of the graph neural network. The graph neural network generates a representation for the target node that is transformed by a feed forward neural network tower into an output vector.

Embodiments of the present disclosure provide a data processing method and device, electronic equipment and storage medium. The method comprises: acquiring an image, determining a to-be-processed feature corresponding to the image; inputting the to-be-processed feature into an autoregressive sequence generation model to obtain a target feature of the image; wherein the autoregressive sequence generation model comprises a lightweight self-attention subnetwork and a feedforward neural subnetwork; and performing analysis and processing on the image based on the target feature. The technical solution of the embodiments of the present disclosure reduces the requirement of the model on computing power, and even if the model is deployed on a mobile terminal, good data processing effect can be achieved. At the same time, the calculation amount in the model running process is reduced, and real-time processing of related data is facilitated.

The application discloses an image classification system and method based on a lightweight LA Transformer network, comprising a feature extraction module, an LA Transformer main network and an image classification module. The feature extraction module is responsible for local feature extraction of an input RGB image, realizes down-sampling of the input image, and outputs the image as an input of the LA Transformer main network. The LA Transformer main network is responsible for further feature extraction of the image. A local self-attention subnetwork is used to model spatial correlation information between feature map patches and patches by using a local self-attention mechanism, and an attention feedforward neural subnetwork is used to model correlation information between feature map channels and channels. The image classification processing module is responsible for generating a probability for each image classification, and completes classification of the input RGB image. The application can achieve higher image classification accuracy while having lower parameter quantity and calculation amount.

The invention discloses a model loading and unloading method and electronic equipment, and relates to the technical field of server resource management.The method comprises the steps that feedforward neural sub-networks are preloaded to preset hardware layer by layer according to a preset sequence and historical operation process data; performing first calculation on the first resource use information, the second resource use information and the memory use information according to a preset time interval to obtain a first resource use function value; performing second calculation on the second resource use information and the memory use information according to a preset time interval to obtain a second resource use function value; when it is detected that the second resource use function value is smaller than a second predefined resource use function value threshold value lower bound, the feedforward neural sub-network in the running process is unloaded from the first processor to the second processor. According to the invention, the problem of low model feedback efficiency caused by unbalanced distribution of each sub-network in each hardware of the server in the model in the related technology is solved.

This application discloses a model loading / unloading method and electronic device, relating to the field of server resource management technology. The method includes: preloading each feedforward neural subnetwork onto preset hardware in a preset order based on historical operation data; performing a first calculation on first resource usage information, second resource usage information, and memory usage information at preset time intervals to obtain a first resource usage function value; performing a second calculation on second resource usage information and memory usage information at preset time intervals to obtain a second resource usage function value; and unloading the feedforward neural subnetwork from a first processor to a second processor when the second resource usage function value is detected to be less than a second predefined lower bound of a resource usage function value threshold. This invention solves the problem of low model feedback efficiency caused by the uneven distribution of subnetworks in the model across different hardware components of the server in related technologies.

The invention relates to a space-time sequence prediction method, and belongs to the field of artificial intelligence time sequence prediction. The method comprises the following steps: S1, processing a node time sequence by adopting a sliding window sampling method; s2, performing random masking on the samples; s3, capturing a hidden representation of the long-time sub-sequence sample based on a multi-head attention mechanism; s4, obtaining an incidence matrix of the hypergraph based on the nodes and the hyperedge representation matrix; s5, updating the node information; s6, updating the hyperedge information; s7, defining hyperedge and node constraint functions; s8, inputting the short-time sub-sequence samples into a gating recursion unit to obtain hidden representation; and S9, clustering the nodes based on a K-means algorithm, and inputting the hidden representations of the long-time and short-time sub-sequence samples into a feedforward neural layer to obtain a prediction result of the space-time sequence.

The application discloses a model deployment method and device, and the method comprises the following steps: acquiring a pre-trained visual convolution model, setting an accuracy target and search parameters for pruning the visual convolution model, wherein the search parameters at least comprise an initial pruning ratio and a search interval length; searching a first target pruning ratio of an attention mechanism module and a second target pruning ratio of a feedforward neural module respectively; optimizing the visual convolution model according to the first target pruning ratio and the second target pruning ratio to obtain a target visual convolution model; and deploying the target visual convolution model on an edge computing device of a substation. In this way, after the visual convolution model is pruned, the model size is in a minimum state, so that the requirement for the running environment is reduced. The visual convolution model is deployed on the edge computing device, real-time reception of monitoring data is realized, so that real-time detection and warning can be achieved, and the safety of the power system is improved.