56 results about "Attentional network" patented technology

The invention discloses a fine-grained source code vulnerability detection method based on a graph neural network, which comprises the following steps: A, for all source codes of software, extractinga code attribute graph of a function in the source codes, and performing vectorization representation on the code attribute graph, the vectorization representation comprising vectorization representation of node attributes and graph structures; B, extracting code features by using a graph attention network, and processing all statements to obtain a final data set; and C, training a vulnerability detection model, and determining a final model through the effect on the test set. The source code vulnerability detection method provided by the invention has the advantages that the automation degreeis higher; dependence on domain expert knowledge is reduced, code auditing cost is greatly saved, code auditing efficiency is improved, grammar and semantic information in source codes are better reserved, meanwhile, vectorized representation of statements can be learned purposefully through a graph attention network, and vulnerability detection of a finer-grained statement level is achieved.

The invention discloses a vehicle trajectory prediction method based on an environmental attention neural network model, and constructs a model for increasing attention to each element in the environment, namely an environmental attention network (E-ANet) model. According to the model provided by the invention, transverse expansion is carried out on the basis of a structure in which an LSTM encoder and a convolutional social pool are connected in series, and a graph attention neural network and the convolutional social pool containing an SE module are added to form a parallel structure. Through the novel parallel structure, feature information updated by connecting edges of all nodes in a graph structure formed by the vehicle and the surrounding environment in the running process and feature information in a spatial position structure in the surrounding environment are captured. Compared with a convolution social pool model, the new model structure provided by the invention has the advantages that the effect of extracting the environment interaction information is greatly improved, and meanwhile, a better track prediction effect is achieved compared with other existing models.

The invention belongs to the technical field of artificial intelligence, and particularly relates to a user preference prediction method based on a session recommendation system. The method is based on a neural network technology and is divided into two stages. Firstly, based on input session sequence data, a session graph is constructed, and a graph neural network is used for learning vector representation of each item. Secondly, respectively long-term and short-term preferences of the user are learned by using a self-attention network and a pooling network, and recommendation is performed bycombining an attention mechanism with the two parts. The finally obtained model is used for predicting the preference of the user.

The invention discloses a false news detection system and method based on cooperation of a decision tree and common attention. From a new perspective, the invention provides a traditional transparentmachine learning and neural network model combined method, and provides an interpretable false news detection method based on cooperation of a decision tree and common attention, so as to transparently capture powerful fine-grained evidences and explore false parts of false news through the evidences. According to the invention, the false news detection performance is improved, and the transparency of the detection process and the interpretability of the detection result are also provided. According to the method, a transparent and interpretable scheme is provided for a false news detection task, the decision tree model is combined into a common attention network, evidence capable of interpreting claim settlement verification can be provided, and meanwhile the evidence forming process is interpreted through judgment conditions. The system has detachability, two modules of the system can be used for decoupling training. The system has model generalization ability and task staged training ability.

The invention relates to the technical field of image processing, in particular to a GACNN-based multi-label eye fundus image recognition method, which comprises the following steps of: acquiring an original eye fundus image and preprocessing the original eye fundus image; constructing a GACNN model, using the preprocessed original fundus image with the label to perform training, wherein the GACNN model comprises a convolutional neural network, a graph attention network and a fusion layer, and the convolutional neural network is used for extracting image features, the graph attention network is used for modeling a relationship among fundus multiple labels, regarding each label of a fundus image as a group of mutually dependent nodes, and training by using historical data to obtain a multi-label classifier, and the fusion layer fuses the features obtained by the convolutional neural network and the graph attention network to obtain a final classification result; and inputting a to-be-detected original fundus image into the trained GACNN model, and outputting an identification result with a label; According to the method, the correlation between the tags is fully considered when the multiple tags in the fundus image are identified, and the identification accuracy of the fundus image is improved.

A method for training an object detection network by using attention maps is provided. The method includes steps of: (a) an on-device learning device inputting the training images into a feature extraction network, inputting outputs of the feature extraction network into a attention network and a concatenation layer, and inputting outputs of the attention network into the concatenation layer; (b) the on-device learning device inputting outputs of the concatenation layer into an RPN and an ROI pooling layer, inputting outputs of the RPN into a binary convertor and the ROI pooling layer, and inputting outputs of the ROI pooling layer into a detection network and thus to output object detection data; and (c) the on-device learning device train at least one of the feature extraction network, the detection network, the RPN and the attention network through backpropagations using an object detection losses, an RPN losses, and a cross-entropy losses.

The invention discloses a false news detection system and method based on an evidence perception hierarchical interaction attention network, which discusses more credible evidence semantics by capturing evidence fragments and fusing source credibility, and is used for discovering suspicious parts in false news so as to carry out interpretable false news detection. According to the method, suspicious parts in claim settlement can be captured, more credible evidence semantics can be explored for the parts, and the interpretability of false news detection is enhanced; the internal interaction module composed of self-attention and symmetric attention enhances deep interaction and matching between news semantics and related articles, so that key evidence fragments can be effectively obtained; the global reasoning module not only can obtain more credible evidences from key evidence fragments through gating affine transformation, but also can relieve the influence of extreme semantics on a detection result through global interactive reasoning.

The invention discloses a method for solving video question-answer problems by using a multi-granularity convolutional network self-attention context network mechanism. The method mainly comprises thefollowing steps that: 1) for a group of videos, frame-level and segment-level video expressions are obtained by using a pre-trained VGG network and a 3D-Conv network; 2) for question word embedding and answer word embedding of dialogue histories and new questions, the multi-granularity convolution self-attention network mechanism and a sentence-level context attention mechanism are adopted to obtain joint expressions related to the questions; and 3) a question-level time attention mechanism and a fused attention network mechanism are adopted to obtain joint video expressions related to the questions and generate answers to the questions asked about the videos. Compared with a common video question-answer solution, the method is advantageous in that the multi-granularity convolutional self-attention network is utilized, so that visible information and dialogue historical information can be combined to generate answers which better meet requirements. The method achieves a better effectin the video question-answer problems compared with a traditional method.

The invention discloses a graph convolutional neural network action recognition method based on an attention mechanism, and relates to the field of human-computer interaction action recognition. The method comprises the steps of labelling N attention joints with the highest motion completion participation degree by a residual attention network, N can be 16, and setting other numerical values according to the actual situation; constructing a three-dimensional skeleton space-time diagram, and carrying out space-time feature coding on the attention joints; and learning the three-dimensional skeleton space-time graph through a graph convolutional neural network GCN to perform action recognition. The joints with high participation degree for completing the specific action are selected based on the residual attention network, so that the information processing redundancy can be reduced, and joint information which does not contribute to action recognition is abandoned; based on space-time constraints between the joints, space-time feature codes about the attention joints are constructed to more effectively represent space-time features of the attention joints; based on human body space structure natural graph representation, a graph convolutional neural network is utilized to obtain depth representation about a three-dimensional skeleton space-time graph so as to effectively recognize actions.

The invention discloses a video question-answering method based on a progressive graph attention network, and the method comprises the steps: sampling a novel progressive graph attention network, andexploring various types of visual interaction at a target level, a video frame level and a video segment level in a progressive mode; in the progressive graph attention network, the graph structure ofthe target level is mainly used for obtaining the space-time relationship between targets in the same frame or different frames, the graph structure of the video frame level explores the mutual relationship between video frames, and the graph structure of the video clip level constructs the time sequence relationship between different actions. Meanwhile, an attention mechanism is also used for paying attention to graph vertexes and edges related to problems, and graph features of different levels are connected in a progressive mode. In this way, each graph can focus on its spatio-temporal contiguous vertices and finer grained visual content based on visual correlations. Therefore, the accuracy of predicting the answers to the questions is improved.

The invention discloses a medical image segmentation method and system based on AS-UNet. According to the invention, an edge attention network framework is provided, edges are enhanced, and missing values are reduced. The method comprises the following steps: obtaining a mask edge image through a mask edge extraction algorithm, and connecting the mask edge image to the last three layers of the UNet expansion path to reinforce edge information; and a new attention module being introduced into the BAB, channel attention and space attention being combined, feature response being activated, acquisition of key information in the image being enhanced, and the target area segmentation capability of the network being improved. According to the method, a region and boundary combination loss function is used, so that the segmentation precision is improved, and meanwhile, parameters are reduced during testing. And network parameters in the AS-UNet are continuously updated through forward and backward feedback during training under the action of the combined loss function, so that the trained model can abandon the added parameters of the BAB part during testing, and the time cost of prediction is reduced.

The invention relates to a human body action recognition method based on a separable three-dimensional residual attention network, and belongs to the field of computer vision. The method comprises the following steps: S1, replacing a standard three-dimensional convolution in a 3D ResNet with a separable three-dimensional convolution, and constructing a Sep-3D ResNet; S2, designing a channel attention module and a space attention module, and stacking the modules in sequence to construct a dual attention mechanism; S3, carrying out double attention weighting on the middle-layer convolution features at different moments, a double attention module is expanded in the time dimension, then the double attention module is embedded into a Sep-3D RAB of the Sep-3D ResNet, and a Sep-3D RAN is built; and S4, carrying out joint end-to-end training on the Sep-3D RAN by utilizing a multi-stage training strategy. The method can improve the distinguishing capability of classification and discrimination features, achieves the efficient extraction of high-quality space-time visual features, and can enhance the classification precision and recognition efficiency of the model.

The invention provides a three-dimensional object recognition method based on a multi-view double-attention network, and the method comprises the following steps: 1, projecting an original three-dimensional object from n views to a two-dimensional plane, rendering the original three-dimensional object to obtain n views, and carrying out the feature extraction of the n views through a basic CNN model, and correspondingly obtaining n visual features; 2, inputting the n visual features into a view space attention module for processing to obtain a visual space descriptor; 3, inputting the n visual features into a view channel attention module for processing to obtain visual channel descriptors; and 4, combining the visual space descriptor and the visual channel descriptor to obtain a three-dimensional shape descriptor, and inputting the three-dimensional shape descriptor into a full-connection network to complete object identification so as to obtain a prediction identification classification result of the original three-dimensional object.

The invention discloses a pedestrian re-identification method based on a spatial reverse attention network. The pedestrian re-identification method comprises the steps of collecting shot pictures and dividing into a training set and a test set; constructing a spatial reverse attention network model based on Resnet-50, training a convolutional neural network according to the training set, and adding CBAM-Pro; dividing the network into two branches according to the added CBAM-Pro, executing forward learning and reverse attention at the same time, and extracting forward and reverse global features and local features; and connecting the extracted features according to channel dimensions to obtain pedestrian identification features including multiple types of features, and performing re-identification verification on the pedestrian identification features by using the test set to complete pedestrian re-identification. According to the method, multi-type pedestrian identification features are extracted based on the spatial reverse attention network, and the effectiveness and reliability of re-identification are improved.

The invention discloses a dual-channel semantic positioning multi-granularity attention mutual enhancement video question answering method and system. The method comprises the following steps: constructing a feature extraction coding module to extract features; constructing a coarse and fine granularity global retrieval module to perform attention calculation on the features; constructing a graph attention network module to enhance key information between the features; constructing a local positioning module to position key feature information of different channels; and constructing a fusion attention module to perform multi-level fusion on the enhanced features, and finally obtaining answer index information through an answer prediction module. According to the method, feature information of different granularities in a video is defined as visual semantic and text semantic channels through multi-module design, an auxiliary positioning mechanism is designed for different channels, and feature information most related to a problem is obtained by enhancing sharing representation. Correlation mutual attention is carried out on global and local information by utilizing a graph attention network, so that characteristics related to questions under the same time and space are optimally expressed, and the difficulty of current video question answering is relieved.

The invention discloses an indication expression understanding method based on a multi-level expression attention-guiding network, and innovatively designs a new multi-level attention mechanism, namely a multi-level expression attention-guiding network (MEGA-Net), which comprises a three-level attention network. The multi-level attention mechanism can generate image region representations with distinction degrees under the guidance of expression representations of different levels (statement levels, word levels and phrase levels), thereby helping to accurately determine a target region. In addition, an existing method generally adopts a single-stage mode to match regions, and the mode cannot well distinguish similar objects or targets. Aiming at the problem, the invention designs a two-stage structure to compare the similar image areas and find out the difference between the similar image areas, so as to match the optimal image area. According to the method, evaluation is carried out on three popular data sets, and experimental results show that the performance of the method is superior to that of other highest-level models.

The invention discloses an image description method of a self-attention mechanism based on sample adaptive semantic guidance, and belongs to the technical field of artificial intelligence. To solve the defect that parameters of each sample are fixed in a traditional method adopting a self-attention mechanism, the method comprises the following steps: 1) extracting features corresponding to a plurality of candidate areas of a to-be-described image by a target detector; 2) extracting a plurality of semantic concepts for the to-be-described image by adopting a semantic concept detector; 3) performing feature enhancement on the features extracted in the step 1) and the step 2) through different self-attention networks; 4) generating a parameter of a self-attention network by using the semantic concept feature enhanced in the step 3) and a parameter generation network; 5) inputting the visual features enhanced in the step 3) into the generated self-attention network, and realizing better visual expression through the self-attention network generated semantically; and 6) inputting the visual features output in the step 5) into a decoder, generating a description statement of the image, and defining a loss function.

The invention discloses a self-attention network information processing method for streaming media recommendation. The method comprises the following steps: injecting a position embedding matrix intoan input embedding matrix to obtain a final input embedding matrix; normalizing the final input embedded matrix to obtain a linear projection onto the three matrixes to obtain Q, K and V, and obtaining a weight to obtain an output H1 representing interaction between items in the session in the first self-attention; performing self-attention splicing on the outputs representing the interaction between the items in the session to obtain an interaction S representing the final item in the session; summing the E and the S, performing layer normalization processing to obtain a matrix F representinguser preferences through feedforward network processing, and obtaining each layer of matrix representing the user preferences by applying a nonlinear activation function on the basis of multi-head self-attention; and calculating the score ri of the candidate item vi belonging to V, and determining the probability that the user clicks the candidate item vi according to ri.

The invention discloses an image segmentation method based on a reverse attention network, and the method comprises the steps: firstly obtaining an image data set, and constructing a training set and a test set; then, creating a reverse attention network model, wherein the processing process of the reverse attention network model comprises the steps that the image is coded layer by layer through a plurality of convolutional layers to sequentially obtain output feature layers of different levels, and the output feature layers of the different levels are spliced and aggregated in parallel and then input into a decoder to be decoded to obtain a global feature image; inputting the global feature image and the output feature into a reverse attention network for processing until a low-level reverse attention feature is obtained; and inputting the training set into a reverse attention network model for training to obtain a trained reverse attention network model, and obtaining an image segmentation result. Advanced features obtained by image coding are input into the reverse attention network, so that the precision of image segmentation is greatly improved.

The invention discloses a point cloud local feature extraction method, device and equipment and a storage medium. The method comprises the following steps: firstly, providing a new graph attention network for point cloud local feature extraction, and enabling the network to rapidly and accurately carry out the semantic segmentation of a three-dimensional point cloud; secondly, constructing a local expansion graph area of each point by using an expanded K nearest neighbor search algorithm, and performing local feature expression by using Euclidean distance geometric correlation between a central point and a neighborhood thereof; finally, applying an attention mechanism to a designed network layer, called as a graph attention layer, dynamically learning the context attention features on the local expansion graph by giving proper weights to adjacent edges of a central point, and better reserving local geometric details of the point cloud through attention pooling operation. Compared with an existing point cloud local feature extraction method, the method achieves better performance in three-dimensional point cloud shape classification and segmentation tasks.

The invention discloses a fashion compatibility analysis method and system based on deep multi-modal feature fusion, and the method comprises the steps: processing text data through a sample feature extraction network, a visual feature extraction network based on Resnet-18, and a text feature extraction network based on one-hot coding; after features are extracted, the features are fused, a visual feature and text feature fusion network based on an attention mechanism, extracted visual features and text features are fused, a visual feature self-attention network based on the attention mechanism is adopted, and feature expression of a visual mode is enhanced; mapping the fusion features into a multi-modal vector space by using a feature representation network based on multi-layer mapping; and finally, a fusion feature compatibility calculation network is used, a fusion feature positive pair distance is shortened in a multi-modal vector space, and a negative pair distance is expanded. According to the method and the device, the fashion single items can be reasonably matched, and the accuracy of the fashion single item matching result is improved.

The invention discloses a short video classification method based on multi-modal feature complete representation, and the method comprises the steps: for the content information of a short video, providing constructing four subspaces from the perspective of modal missing mainly based on a visual modal feature, and obtaining potential feature representations respectively, further fusing the potential feature representations of the four subspaces by using an automatic coding and decoding network to ensure that more robust and effective public potential representations are learned; for label information, using inverse covariance estimation and a graph attention network to explore correlation between labels and update label representation to obtain label vector representation corresponding to the short video; providing a multi-head cross-modal fusion scheme based on multi-head attention for the public potential representation and the label vector representation, wherein the multi-head cross-modal fusion scheme is used for obtaining a label prediction score of the short video, wherein the overall loss function of the model is composed of traditional multi-label classification loss and reconstruction loss of an automatic coding and decoding network and is used for measuring the difference between a network output value and an actual value and guiding the network to find an optimal solution of the model.

The invention discloses a facial pain expression recognition method and system based on depth information. The method comprises the following steps: establishing a pain expression data set containing the depth information; preprocessing the acquired pain expression data set; inputting the preprocessed pain expression data set into a frame attention network for training to obtain a pain grading model, wherein the frame attention network comprises a feature embedding module and a frame attention module; and sending expression data which are acquired in real time and preprocessed to the pain grading model for identification, and judging whether to carry out alarm processing according to an identification result. According to the invention, abnormal state detection can be stably carried out on the identified target, early warning can be timely sent to a worker, and the method has great practical value in the fields of public medical treatment, smart old-age care and the like.

The invention discloses a traditional Chinese medicine syndrome type identification method based on a graph attention network, and the method comprises the steps: standardizing a plurality of medical case data, and constructing a training set, namely corpus data; respectively establishing a symptom set and a syndrome set for all non-repeated symptoms and syndromes in the corpus; every two symptoms in each medical case data belonging to the training set being connected to serve as nodes, calculating point mutual information between the two symptoms in the symptom set, and updating nodes in a graph by using a graph attention network; weighting the updated nodes by using an attention mechanism to obtain feature vectors of symptoms; inputting the feature vectors into a linear layer for classification to obtain the probability of each syndrome; calculating a loss function in combination with the probability of each syndrome type and the real condition in the training set; performing back propagation according to the loss function, and completing model iteration; and inputting symptom information to be identified into the trained model to obtain a symptom type identification result. According to the invention, the accuracy of identification of the syndrome type is effectively improved.

The invention provides a question and answer matching method based on an adaptive fusion multi-attention network, and the method comprises the following steps: S1, converting each word in a question and an answer into a word vector, and coding the word vectors to obtain a word vector sequence; s2, extracting information of different aspects in a problem by using a plurality of self-attention networks, and respectively encoding the information into different problem vectors; s3, generating a corresponding answer vector for each question vector; and S4, calculating a matching degree score, adaptively fusing the matching degree scores of the information in multiple aspects, and carrying out question and answer matching according to the fused matching degree score. The invention provides a question and answer matching method based on an adaptive fusion multi-attention network, and solves the problem that a model using a plurality of self-attention networks at present enhances understanding of questions and answers through multi-round processing instead of obtaining multi-angle and multi-aspect information of the questions and answers.

The invention provides a traffic flow prediction method fusing a space-time attention neural network and a traffic model, and the method comprises the steps: dividing feature data according to time slices, carrying out the GAT operation of the data on each time slice, and obtaining a new representation of a flow feature and a speed feature, the new representation of the speed characteristics is input into a traffic simulation model Greenshifts parabola model for transformation to obtain another new representation of the flow characteristics, then the two new representations of the flow characteristics are respectively subjected to gated loop unit network GRU processing, and then the two obtained flow representations are spliced to obtain input of a full connection layer; and performing full connection layer processing on the spliced feature data to obtain a final prediction result, and finally training a neural network model based on a deep learning theory. And obtaining a prediction result on the test set by using the trained network model. According to the method, the traffic flow prediction in the future time period can be realized under the condition that the traffic network and the flow characteristic and speed characteristic data thereof are known.

The invention provides a contract document fault-tolerant information extraction method based on a graph attention network, and relates to the technical field of computers and information processing. The method comprises the following steps: firstly, performing character recognition on a contract through an OCR (Optical Character Recognition) engine to obtain text content and corresponding position coordinates; text information features are extracted, wherein the text information features comprise position vectors of text information and word embedding expressions of text character strings; taking features extracted from the contract document as graph node features, and constructing a fault-tolerant contract text relation graph; then setting each layer structure and an activation function of the graph attention network; inputting the training set into the constructed graph attention network for training until the loss function converges; and finally, modeling a contract to be recognized into a text relation graph, inputting the text relation graph into the trained graph attention network, and finally obtaining the category of text information. According to the method, the dislocation information extraction of the contract document is realized, and the method has higher recognition efficiency and accuracy compared with the existing information extraction technology after OCR, and is beneficial to office intelligentization.

The invention discloses a reading understanding method based on an ELMo embedding and gating self-attention mechanism. The method is based on a model involving an ELMo embedding and gating self-attention function, adopts ELMo representation, character representation of words and multi-dimensional fusion representation of glove word representation simultaneously, and introduces a self-attention layer with a gating function behind a bidirectional attention network to extract the long context information and further to filter the information. Besides, in the answer layer, the method multiplexes the feature representation obtained in each layer, and uses a bilinear function to predict the position of the final answer, thereby further improving the overall performance of the system. In an experiment on an SQuAD data set, it is proved that the model is greatly superior to many baseline models, the performance of the model is improved by about 5% compared with an original baseline of the model, the performance of the model is close to the average level of a human test, and the effectiveness of the method is fully proved.

The invention discloses a complex coal mine image defogging method based on an improved generative adversarial network, and the method comprises a two-stage network: a combined defogging network based on a haze imaging model, and a refinement network based on the generative adversarial network. The first-stage learning is realized by directly embedding a haze imaging model into a network, so that the proposed method is ensured to strictly follow a physical scattering model for defogging. And for the second stage, a conditional generative adversarial network (GAN, Generative Advanced Network) is provided to recover background details which cannot be captured in the first stage, and the image artifacts introduced in the stage are corrected. The invention provides a novel channel attention network (CA, Channel Attention), which is inspired by the attention mechanism and is used for recovering a real and clear image from the output result of the first stage. In order to obtain a better result, a basic GAN formula is further modified, and perceptual fusion is introduced.

A method and apparatus include receiving a text input that includes a sequence of text components. Respective temporal durations of the text components are determined using a duration model. A spectrogram frame is generated based on the duration model. An audio waveform is generated based on the spectrogram frame. Video information is generated based on the audio waveform. The audio waveform is provided as an output along with a corresponding video.