175results about How to "Enhance expressive ability" patented technology

The invention provides a method for fusing a full-color image and a multispectral image based on a deep neural network. Specific steps are as follows: step 1. constructing a training set of high resolution and low resolution image block pairs; step 2. using an improved sparse denoising self-encoder to learn to train initialization parameters of a first layer in a neural network model; step 3. using the improved sparse denoising self-encoder to perform pretraining of the neural network layer by layer; step 4. performing fine adjustment of parameters of the pretrained deep neural network; and step 5. using the deep neural network to reconstruct a multispectral image of high resolution according to a known multispectral image of low spatial resolution. The method provided by the invention adopts a method of deep learning, and can make full use of a nonlinear neural network to depict complex structural information of a multispectral image, thereby enabling the fused multispectral image to have high spatial resolution, and well keeping spectral information of the multispectral image.

The invention discloses a natural scene text detection method based on full convolution neural network, This method uses CNN network to extract the feature representation of text, and adjusts the feature representation by feature fusion module, at the same time fuses the semantic feature of high-level feature map and the position information of low-level feature map, so that the extracted featurehas stronger representation ability, and combines with text prediction module to directly predict the candidate text object. This method adopts end-to-end training and prediction process, The processing flow is simple, and no multi-step hierarchical processing is needed. Finally, the final detection result is obtained by simple NMS operation, which has high accuracy and strong robustness. The multi-directional and multi-size text object in the natural scene image with complex background can be detected well, and the detection performance of the natural scene text is excellent.

The invention provides a method and device for detecting video abnormal events. The method includes the steps that high-level expression information of a to-be-detected video stream with multiple images is extracted, wherein the high-level expression information comprises space-time information of the to-be-detected video stream; through a preset dictionary, reconstruction coefficients generated when bases with the minimum number in the dictionary are used for representing the high-level expression information of the to-be-detected video stream are calculated; according to the reconstruction coefficients, reconstruction cost values are calculated; when the reconstruction cost values are larger than a preset threshold value, it is confirmed that the abnormal events exist in the to-be-detected video stream; when the reconstruction cost values are smaller than or equal to the preset threshold value, it is confirmed that no abnormal events exist in the to-be-detected video stream. By the adoption of the method, the feature expression capacity is high, the abnormal events can be well described, and the detection efficiency and the detection accuracy of the video abnormal events are improved.

The invention relates to the technical field of image super-resolution processing, in particular to a non-negative matrix factorization-based face super-resolution processing method. The method comprises the following steps: performing face alignment on high-resolution face images in a sample library, reading the aligned sample image library, utilizing a non-negative matrix factorization algorithm to perform a factorization operation to obtain a basic image W, performing alignment on input low-resolution face images to obtain the non-negative matrix factorization expression coefficient e of a target high-resolution face image, obtaining the target high-resolution image Z1=We in combination with the basic image W and the expression coefficient e and dividing the important areas of the face images in the sample library; performing factorization synthesis on the divided local areas; and weighting and combining the synthesized local area and the image Z1 to obtain a super-resolution image Z2. The method has the advantages of increasing semantic constraint like that the grayscale of the image is non-negative, improving the expression capacity of the characteristic basic image and finally improving the quality of the super-resolution image.

The invention relates to a front multi-vehicle tracking method integrating a millimeter radar and depth learning vision. The millimeter radar is used for obtaining front data information, according tothe echo reflection strength and width information, invalid information is removed, and only front vehicle information remains. According to the method integrating the millimeter radar and a camera,by filtering radar information and generating a motion trajectory through an online tracking model, and trajectory correlation is performed. The front vehicles related to the trajectory are recorded and numbered. According to the generated trajectory and the numbered front vehicles, the steps just need to be repeatedly executed on data of the next period, consistency checking is performed, and thedata is added into the numbered trajectory. For newly appearing vehicles, trajectory generation, trajectory collection and numbering are performed according to the beginning steps. By combining advantages of the millimeter radar and the vision depth learning, and the target tracking accuracy and robustness for front multiple vehicles can be effectively improved.

The invention discloses a fast multi-label picture retrieval system and a realization method. The method comprises the following steps: deploying an RPN (Region Proposal Network) for extracting region proposals in a convolutional neural network, extracting region proposal information of pictures, and performing ROI pooling calculation on the region proposal information; after pooling, building a multi-label classification loss function through a fully connected layer according to multi-label information to train the convolutional neural network, and building a weighted three-dimensional loss function to train the convolutional neural network; extracting the hash code of each picture from a picture candidate set through the convolutional neural network after multi-task learning, saving the hash codes to a database, and comparing the hash codes with the hash codes in the database, thus completing picture retrieval. The whole network is trained through multi-task learning of classification and hashing, and therefore, the accuracy of retrieval is ensured. Moreover, the similarity is measured using Hamming distance in the process of retrieval, and the efficiency of retrieval is improved greatly.

The invention relates to an image super-resolution method based on a channel attention mechanism and multilayer feature fusion, and the method comprises the steps of directly extracting the original features of a low-resolution image at the beginning of a residual branch by using a single-layer convolutional layer based on deep learning; using six cascaded convolutional circulation units based ona channel attention mechanism and multi-layer feature fusion to extract accurate depth features; carrying out upsampling on the depth features through a deconvolution layer, and carrying out dimensionality reduction on the upsampled features through a single-layer convolution layer to obtain a residual error of the high-resolution image; carrying out up-sampling on the low-resolution image by using a bicubic interpolation method in a mapping branch to obtain mapping of the high-resolution image; and adding the mapping and the residual of the high-resolution image pixel by pixel to obtain a final high-resolution image. The method is reasonable in design, fully considers the difference between the feature channels, efficiently utilizes the hierarchical features, and maintains a higher operation speed while obtaining higher accuracy.

The embodiment of the invention provides a method and a device for training a graph neural network model used for characterizing a knowledge graph, and the method comprises the steps: obtaining a triad from the knowledge graph, and the triad comprises a first node, a second node, and a first connection edge pointing to the second node from the first node; then, in an edge embedding layer, determining a corresponding first edge vector according to the relationship type corresponding to the first connecting edge and the edge attribute characteristics; in the node embedding layer, the first nodeand the second node serving as target nodes respectively, conducting multi-level vector embedding according to the node attribute characteristics of the target nodes and a neighbor node set of the target nodes, and therefore a first high-order vector and a second high-order vector corresponding to the first node and the second node are obtained respectively; then, according to the first high-ordervector, the second high-order vector and the first edge vector, determining the probability that the first node is connected to the second node through the first connecting edge, and updating the edge embedding layer and the node embedding layer with the maximization probability as the target;

The invention discloses a cross-modal image text retrieval method of a hybrid fusion model. The method comprises the following steps of: in an early fusion structure, firstly combining local visual area features and original global features of a text to obtain a unified cross-modal fusion representation, and then taking the fusion features as input, enhancing interaction between the local visual features and the language information in a subsequent embedded network; meanwhile, on the basis of a traditional late fusion structure, inputting an original image and sentence features into a vision encoder and a text encoder respectively for intra-modal feature enhancement, and enriching semantic information of respective modals; and finally, the whole network similarity is a weighted linear combination of early fusion similarity and late fusion similarity so that complementation of early fusion in a cross-modal learning layer and late fusion in an intra-modal learning layer is realized, and potential alignment between image and text modals is completed.

The invention provides a multi-modal pre-training model training method, and an application method and a device thereof. The multi-modal pre-training model training method comprises the following steps: constructing a multi-modal pre-training model of a double-tower structure; obtaining a positive sample data set comprising positive sample image-text pairs and a negative sample data set comprising negative sample image-text pairs; and training a multi-modal pre-training model according to the positive sample data set and the negative sample data set, wherein the multi-modal pre-training model comprises a cross-modal comparison learning module used for carrying out image-text similarity comparison learning on the positive sample image-text pair and the negative sample image-text pair. The multi-modal pre-training model adopts a double-tower structure and a cross-modal contrast learning algorithm, a large number of negative samples are constructed for image and text modals, the model expression ability is high, and the processing precision of image-text pairs is improved. According to the model, the overall similarity between images and texts is calculated, whether the images and texts correspond or not is judged according to the similarity, and on the basis of the image-text weak correlation hypothesis, the actual situation of semantic weak correlation between the images and texts in image-text pairs in actual application is better fit.

The invention discloses a drawing image retrieval method. The method comprises the following steps of constructing a drawing image training sample set, wherein the sample set comprises a plurality of drawing image training samples corresponding to the same drawing works, and the image type of each drawing image training sample in the sample set is marked according to fine classification and rough classification; based on a depth convolution neural network, constructing a drawing image feature extractor, automatically learning fine classification features and coarse classification features from the training sample set, obtaining drawing image feature vectors and storing the drawing image feature vectors into a drawing image feature database; by utilizing the constructed drawing image feature extractor, extracting the feature vector of a to-be-retrieved drawing image; calculating the distance between the feature vector of the to-be-retrieved drawing image and each drawing image feature vector in the drawing image feature database; and obtaining an image retrieval result based on the distance. According to the invention, the accuracy, the accuracy, the robustness and the usability of the drawing image retrieval system are improved.

The invention discloses an attention mechanism and hash-based image retrieval method and device and a storage medium, and the method comprises the steps: obtaining the discrete hash code of each imagebased on a pre-trained attention mechanism-based hash feature extraction model for each image in an original data set, and building an image feature library; obtaining a discrete hash code of a to-be-detected image based on a pre-trained hash feature extraction model based on the attention mechanism; and querying data closest to the discrete hash code Hamming distance of the to-be-detected imagein an image feature library, wherein the image corresponding to the data is a retrieval result. The Hash feature extraction model based on the attention mechanism gives full play to the ability of thedeep convolutional neural network to extract the abstract semantic features of the image; and by embedding the attention module, the network can focus on the visual content capable of expressing semantic information of the whole image as much as possible in the image, the expression effect of hash coding is improved, and image retrieval is more accurate and faster.

The invention discloses a focal plane polarization image denoising method based on a deep neural network. The method comprises the following steps: creating a polarization image data set; designing adeep neural network PDRDN, wherein the network learns a residual image between the noise image and the noiseless image; optimizing the network parameters by using a customized loss function until themodel parameters are stable; and inputting an image to be denoised into the trained network to obtain an output residual image, and then predicting a clean image by using the noise image and the residual image. The method can achieve a better denoising effect, is used for improving the quality of a polarization image, and reduces the impact on polarization information from noise.

The invention provides a machine vision and millimeter-wave radar fused multi-vehicle target tracking method, which comprises the following steps of: acquiring road target information by using millimeter-wave radar, and screening vehicle targets according to a kinematics parameter related filtering model; detecting vehicles in front of the road by using the visual information, and performing multi-vehicle target tracking based on a detection result; projecting the vehicle target into the image by adopting a machine vision and millimeter wave radar fusion model, setting an association judgmentstrategy to associate the visual tracking target with the vehicle target, and correcting the position and the size of the visual tracking bounding box in the image based on distance information detected by the millimeter wave radar. The technical problem that in the prior art, when multiple vehicles in front are continuously tracked, after errors are accumulated due to the fact that the size of avisual tracking boundary frame is too large or too small, an effective target is lost can be solved.

The invention relates to a video target tracking method based on twin network fusion multi-template features, and provides a semi-supervised template online updating strategy, when a to-be-tracked target in a video sequence has complex conditions such as occlusion, deformation and illumination change, the target change and the occluded condition are evaluated by calculating an APCE value and template similarity, when the appearance of the target is greatly changed, the features extracted from the previous frame of picture are fused with the original template features to obtain a new template with higher expression capability, so that the method is favorable for adapting to various complex conditions; in order to improve the generalization ability of the model and adapt to multiple types oftargets, a regularization technology is adopted in the training process to prevent model overfitting; in order to further improve the speed of the algorithm, only an original template is adopted fortracking in a non-complex situation, so that the calculated amount is greatly reduced, and the method provided by the invention achieves a higher running speed than other methods under the condition of obtaining better tracking performance.

The invention discloses a short-term prediction method for marine environment monitoring factors. According to the method, an autoregressive model is adopted for a short-term prediction of marine environment input factors, and then a deep neural network method is adopted for a short-term prediction of output factors. When the characteristics of the autoregressive model and the characteristics of a deep neural network model are efficiently used, short-term input predicted values are obtained by means of the autoregressive model, then predicted values of the output factors are obtained by means of the deep neural network model, and thus the limitations that prediction of a traditional neural network model to time series data is insufficient in extracted substantive characteristics and limited in gradient descent are overcome. The method achieves the prediction of the marine environment monitoring factors, and has an important significance in a short-term forecast of the marine ecological environment, a short-term pre-warning of marine disasters and the like.

The invention relates to the technical field of threat intelligence and discloses a safety device threat intelligence sharing method based on a lightweight field body. The safety device threat intelligence sharing method comprises the following steps that 1, the lightweight body is adopted as an inter-device information communication medium, and a threat intelligence general-field body is established; 2, an intelligence producer localizes the threat intelligence general-field body, obtains original threat intelligence information from network space and converts and maps the original threat intelligence information into lightweight body knowledges; 3, an intelligence forwarding person forwards the lightweight body knowledges to an intelligence user based on body communication service; 4, the intelligence user converts and adapts the received lightweight body knowledges into locally dedicated strategy descriptions acting on operation of network space. The safety device threat intelligence sharing accuracy is improved, the expansibility, content correlation, interface opening and concept consistency capability of inter-device threat intelligence interaction is improved, and the threat intelligence sharing efficiency is improved.

The invention relates to a screen content image quality evaluation method based on edge and structure characteristics, which comprises the following steps of S1, converting an input image from an RGBcolor space to an LMN color space, then acting an imaginary part of a Gabor filter on an L component, and extracting the edge characteristics of the image; S2, converting the input image into a grayscale image, calculating a gradient image on the grayscale image by using a Scharr filter, calculating an LBP image on the gradient image by using a local binary pattern (LBP) operator, and calculatingto obtain structural characteristics of the image on the gradient image according to the LBP image; and S3, according to the two features in the step S1 and the step S2, training an image evaluation model by using a random forest, and predicting the quality scores of all the test images by using the trained model. According to the method, the quality of the image is evaluated by considering the characteristics of a human vision system and combining the edge and structural characteristics of the image, so that the quality evaluation performance of the non-reference image can be remarkably improved.

The invention discloses a gesture image key frame extraction method based on deep learning. The method comprises the following steps: reading an input gesture video, and converting the input gesture video into a video frame image; detecting a gesture in a video frame image by adopting a Mobilenet-SSD target detection model, and segmenting the detected gesture; and training a gesture segmentation image by adopting a VGG16 training model to obtain a corresponding abstract feature, calculating a spatial gradient, and setting an appropriate threshold value to judge a key frame according to a gradient difference between two adjacent frames of images. The Mobilenet-SSD target detection model is used to detect and segments a hand area, background area noise is removed; hand abstract features areaccurately extracted by utilizing VGG-16, so that the expression capability of the picture is greatly enhanced;, the parameter quantity is reduced, the model complexity is reduced, and the method is suitable for small-amplitude changing video key frame extraction.

The invention relates to a heart disease risk diagnosis method based on a deep convolutional neural network model. The method comprises the following steps: building a heart disease risk prediction model based on a deep convolutional neural network, and carrying out training, learning and testing; acquiring an electrocardiogram and a cardiogram of a user by using a wearable device; comparing the electrocardiogram and the cardiogram of the user, and identifying electrocardiogram feature points and cardiogram feature points of the specific event occurrence time of the cardiac impulse cycle; calculating a time interval value of any two feature points to obtain a plurality of feature values, and calculating a plurality of physiological indexes according to the plurality of feature values; outputting the heart disease risk probability corresponding to the physiological indexes through a DCNN model according to each input physiological index; calculating a heart disease risk comprehensive probability value of the user by adopting a formula; and judging the heart disease risk level of the user according to the heart disease risk comprehensive probability value. According to the invention,the risk probability of cardiac acute diseases of the user can be predicted in real time.

The invention discloses a commodity recommendation model for relieving data sparsity and commodity cold start, relates to the technical field of interest mining, and provides a CTR prediction model GEARec for personalized recommendation of commodities. By introducing a graph embedding technology, the problems of sparsity and cold start are solved, pre-trained Embedding is input into an upper-layer deep neural network, the convergence speed of the network can be increased, and the model performance is further improved. The GEARec model is constructed by a pre-trained Embedding layer, a multi-head self-attention network layer, an AUGRU and an MLP. After the input Embedding vector representation of the user behavior sequence passes through the multi-head self-attention network layer and the AUGRU, the calculation result and other Embedding are spliced and input into the MLP to automatically learn the nonlinear relation between the features, and finally the probability that the user purchases candidate commodities is output.

The invention provides a zero sample image classification method and system based on a convolutional neural network and a factor space, and the method comprises the steps: building a unified zero-sample classification neural network: firstly, extracting image features in a data set through a classical convolutional neural network, and enabling the image features to serve as the input of the neuralnetwork; the dimensionality of known factors is reduced by using a factor pressure reduction technology, and the known factors and potential factors are embedded into a network to serve as an intermediate layer to jointly determine a final classification result; the network enables image input to final category output. And training a zero sample classification network, and iteratively determiningnetwork model parameters. And identifying the image by using the zero sample classification neural network to finish classification of the zero sample image. According to the method, a convolutionalneural network model is used for uniformly processing the relationship among the visual space, the factor space and the category space, the problem that the generalization ability of specific linear or nonlinear function expression is not high is solved, and the factors serving as auxiliary knowledge are embedded into the network and are easy to understand, train and use.

The invention relates to a rapid target detection method based on adaptive convolution, and the method comprises the steps: obtaining image data with a label, and forming a training set; constructinga target detection network based on an adaptive convolution module; training the proposed target detection network on the formed training set until the target detection network converges to obtain a trained target detection network; and detecting the image by using the trained target detection network, removing repeated results, and outputting a final result. By dynamically adjusting the parameters of the convolution filter, the expression capability of the detection network for extracting the features is improved, the irrelevant interference features are inhibited, the feature response of theinterested target is enhanced, the extracted features are more suitable for the detection requirement of the current scene, and the detection performance of the extracted features on the small-scaletarget in the complex scene is improved.

The invention provides a lane line segmentation detection method. The method comprises: acquiring a to-be-processed image containing lane lines; inputting the to-be-processed image into a lightweightconvolutional neural network for segmentation processing to obtain a binary image about a lane line; carrying out edge detection on the obtained binary image to obtain a lane line contour; carrying out Hough transform on the obtained lane line contour to obtain a coordinate point set of end points of a Hough straight line of the lane line; and solving the slope of each Hough straight line according to the coordinate point set, dividing lane lines according to the slopes, and obtaining the lane line to which the vehicle belongs by utilizing an intercept screening method. By applying the embodiment of the invention, after the binary image corresponding to the to-be-processed image is output through the lightweight convolutional neural network, the left lane line and the right lane line to which the vehicle belongs are screened by using the intercept method based on the binary image, so that an ROI region does not need to be set, and the robustness of the algorithm is further improved.

The invention relates to a construction method and a fermenting method of antibiotic-resistance-free recombinant bacillus subtilis for expressing glutamate decarboxylase and belongs to the technical field of bioengineering. The construction method includes taking bacillus subtilis WB600 as an original strain, and knocking out a D-alanine racemase gene on a chromosome of the bacillus subtilis WB600 so as to obtain D-alanine deficient WB600 (dal); fusing an optimized gad gene with an antibiotic-resistance-free expression vector pUB110 (a antibiotics resistance gene is replaced by the D-alanine racemase gene) through an overlap extension PCR technology to obtain a polymer, transforming the polymer into competence of the bacillus subtilis WB600, and enabling the polymer to recombine in a host so as to obtain a recombinant plasmid Pub-HpaII-P43-gad-dal, which is named as bacillus subtilis SK44.001 with the preservation number being CCTCC NO:M 2016774. The fermentation liquor enzyme activity of the antibiotic-resistance-free recombinant bacillus subtilis can be up to 8.6 U/mL, and accordingly the antibiotic-resistance-free recombinant bacillus subtilis has significant industrial application value.

The invention belongs to the technical field of knowledge maps, and discloses a knowledge map information representation learning method, a system, equipment and terminal, and the knowledge map information representation learning method comprises the steps: carrying out the preprocessing according to a path constraint resource distribution method; calculating the reliability of all paths, and outputting the reliability to a training set and a test set; initializing the model and setting parameters; generating a triple according to an iterator, and randomly replacing head and tail entities; calculating a loss function of the triple according to the score function; calculating a loss function of an additional path according to the path reliability; performing parameter optimization by using an Adam method; and performing model verification by using entity prediction and relation prediction. According to the method, rich path information contained in the knowledge graph is considered, the modeling effect of entities and relationships is improved, the modeling of the relationships can be optimized by inputting vectors into a complex plane and using rotation to represent the vectors, and the method can be used for link prediction and recommendation systems.

The invention provides a control device which adopts the correlated and parallel state machine mode to control the multi-group function of the communication device in the communication device of the communication system; the control device controls the correlation operation of a plurality of sub state machines according to the correlations of the sub state machines; each sub state machine is corresponding to a group of independent function in the communication device. The control device includes a judging device which judges if the correlation matched with the current sub state machine exists before current sub state machine processes the corresponding processed event, a first event processing device used for processing the processed event corresponding to the correlation to realize the correlation operation of the sub state machines if the correlation exists and a second event processing device used for processing the processed event corresponding to the current sub state machine if no correlation is available; the invention also provides a method which adopts the correlated and parallel state machine mode to control the multi-group function of the communication device. The control device and the method which adopts the correlated and parallel state machine mode to control the multi-group function of the communication device has the advantages of reducing the complexity of the communication device system, uneasy malfunction of the communication device and easy maintenance.

The invention discloses an unsupervised remote sensing image change detection method, a storage medium and computing equipment. The method comprises the following steps: constructing a multi-scale image convolutional neural network; respectively inputting the dual-temporal images into a multi-scale image convolutional neural network, extracting spatial features and inter-spectrum features, and jointly calculating to generate an initial pseudo label; cascading the two images of the dual-temporal image, inputting the two images into a multi-scale image convolutional neural network, and trainingthe multi-scale image convolutional neural network to generate a two-channel difference image; utilizing a metric learning module of the multi-scale graph convolutional neural network to update the initial pseudo label as a label of a two-channel difference graph, and training the generated two-channel difference graph; and comparing two channels of the trained two-channel difference image to obtain a binary change image with the same size as the original image, and completing image change detection. According to the method, the change detection graph of the pair of dual-temporal images can beefficiently and accurately obtained in an unsupervised manner.

The invention discloses a polarized SAR image classification method based on ACGAN, and the method comprises the steps: carrying out the Pauli decomposition of a polarized scattering matrix, and building a feature matrix based on pixel points; replacing each element in the feature matrix with an image block of a neighborhood of the element to obtain a feature matrix based on the image block; constructing a training data set by using the feature matrix based on the image blocks, and training the ACGAN network model by using the training data set to obtain a pixel-level classification result; and finally, converting the feature matrix into an RGB pseudo-color image, and dividing the image into K super-pixel areas by using an SLIC super-pixel algorithm. And a final classification result is optimized by combining the pixel-level classification result and the super-pixel block. According to the method, the polarization scattering information and the spatial neighborhood information of the polarization SAR data are fully utilized, and the generative adversarial network with the auxiliary classifier is used for competitive adversarial training, so that the classifier can extract classification features more effectively, and higher classification precision is obtained.

The invention belongs to the technical field of biological recognition in the field of information security, and particularly relates to a face recognition method and device based on facial veins. Theface recognition method comprises the following steps: (1) image acquisition and preprocessing; (2) improvement of a convolutional neural network; (3) model training; (4) a registration stage; and (5) a verification stage: judging whether a feature vector corresponds to a certain face or not according to a cosine distance and a cosine similarity threshold. According to the invention, the facial vein features of the human face are fused with the human face, so that the defect that living body detection is difficult in a common RGB image-based human face recognition method is effectively overcome; and moreover, even if the same-egg twins with very similar facial features have great difference in facial vein distribution, the same-egg twins can be effectively distinguished by the method.