A natural source search method based on SVM improved convolutional neural network

Abstract

The application discloses a natural source search method based on an SVM improved convolutional neural network, and comprises the following steps: S1, collecting different formats of languages, and importing the languages into a conversion program in a plaintext mode; S2, importing mark information in a newly generated language database into a language generation program, analyzing feature data imported into the language generation program by using a pre-constructed deep neural network model, and automatically generating a general functional description language; S3, randomly selecting two literal languages from the general functional description language, learning a common occurrence property between words of each source language, converting the property into a low-dimensional real value distributed representation, and forming a resource dictionary vector; and S4, identifying the resource dictionary vector based on the SVM improved convolutional neural network. The natural source search method based on the SVM improved convolutional neural network increases spatial relation judgment capability and improves search efficiency.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of natural language search, in particular to a natural source search method based on an improved convolutional neural network of SVM. BACKGROUND

[0002] With the continuous development of big data related infrastructure, servers, software systems and theoretical system, the current big data analysis solution has gradually matured and is becoming more and more popular. How to use big data and extract valuable information from it, how to optimize the recommendation algorithm and search engine of the shopping platform, is a new topic for big data. Artificial intelligence emerges as the times require, and deep learning is one of the best ways to solve this problem. The initial deep learning is to use neural networks to solve the learning process of feature expression. Deep learning itself is not a completely new concept, but a deepening of the basic artificial neural network level and model based on the development of artificial neural network.

[0003] Deep learning has been widely applied in many fields such as medicine, meteorology, natural language processing, image classification, search engines, etc. The traditional discipline of image processing and the field of deep learning are mutually integrated, so that the development of many industries can be based on image recognition to improve and bring valuable wealth to human beings.

[0004] For example, patent number CN108960772A discloses an enterprise evaluation auxiliary method and system based on deep learning, which comprises the following steps: step S1: obtaining the enterprise evaluation auxiliary request submitted by the user end; step S2: collecting network data according to the identity information of the enterprise; step S3: performing word segmentation on each text related to the enterprise to be evaluated, and then generating a word vector by using a word2vec tool; step S4: for each text related to the enterprise to be evaluated, inputting the word vector into a convolutional neural network model trained by using data in a natural language text library to realize text classification. The application can help the evaluators to collect and organize information, which is conducive to improving the work efficiency and work quality of the evaluators and reducing the threshold of the evaluators for writing the evaluation. However, in order to improve the training effect of the deep neural network, people make corresponding improvements on the connection mode of neurons, activation function, learning and training algorithm, etc. In this scheme, many design methods in the early stage of the development of neural networks are used. Because the training data is insufficient and the computing capacity is backward at that time, the final effect is not satisfactory.

[0005] At present, no effective solution has been proposed for the problems in the related art. SUMMARY

[0006] In view of the problems in the prior art, the application provides a natural source search method based on an improved convolutional neural network of SVM to overcome the above technical problems of the prior art.

[0007] To this end, the application adopts the following specific technical solutions:

[0008] A natural source search method based on an improved convolutional neural network of SVM, the natural source search method comprising the following steps:

[0009] S1, collect languages in different formats, import them into a conversion program in a plaintext manner, and generate a new language database;

[0010] S2, import the mark information in the newly generated language database into a language generation program, analyze the feature data imported into the language generation program by using a pre-constructed deep neural network model, and automatically generate a general functional description language;

[0011] S3, randomly select two literal languages from the general functional description language, learn the common occurrence properties of words in each source language based on different monolingual corpora of the literal languages by using a loss function, convert the properties into low-dimensional real-valued distributed representations, and form a resource dictionary vector;

[0012] S4, identify the resource dictionary vector by using the improved convolutional neural network of SVM, search for a related dictionary, and list the search results.

[0013] Further, the collection of languages in different formats, the import of the languages into a conversion program in a plaintext manner, and the generation of a new language database further comprise the following steps:

[0014] S11, perform piecewise scanning reading of the imported information, and compare the information with an existing database to confirm the function represented by the information or the current state information of the equipment represented by the information;

[0015] S12, mark the corresponding information function or equipment current state information to the newly generated database, and record the corresponding language.

[0016] Further, the import of the mark information in the newly generated language database into a language generation program, the analysis of the feature data imported into the language generation program by using a pre-constructed deep neural network model, and the automatic generation of a general functional description language:

[0017] S21, map the behavior features imported into the language generation program to a high-dimensional space, and use the high-dimensional space as the input of the deep neural network model;

[0018] S22. Train the parameters of the deep neural network model using the click behavior records imported into the language generation program as training samples;

[0019] S23. Extract the information imported into the language generation program, calculate the correlation between language and language conversion behavior, obtain the correlation score between language and language conversion behavior, and rank them.

[0020] Furthermore, the formula for calculating the correlation between the computational language and language conversion behavior is as follows:

[0021]

[0022] In the formula, y U y represents a distributed vector representing the information extracted from the language generation process after analysis. V This represents a distributed vector of the analyzed language conversion behavior dataset, where cosine() represents the cosine function, and y U T Indicates y U The transpose of the vector, where |||| represents the vector value.

[0023] Furthermore, the loss function includes a first loss function, a second loss function, a third loss function, and a fourth loss function.

[0024] Specifically, the first loss function is used to learn the property of co-occurrence among words in each source language, and this property is transformed into a low-dimensional real-valued distributed representation to generate source language word vectors.

[0025] Using the second loss function, the property of co-occurrence among target language words is learned, and this property is transformed into a low-dimensional real-valued distributed representation to generate target language word vectors;

[0026] A resource dictionary vector is constructed using source language word vectors and target language word vectors;

[0027] The third loss function is determined based on the resource dictionary vector formula;

[0028] Using the fourth loss function, search for synonymous source language words in the source language and incorporate the synonymous source language words into the resource dictionary vector.

[0029] Furthermore, the calculation formula for the resource dictionary vector formula is as follows:

[0030]

[0031] in, They represent source language words and target language words, respectively. express The corresponding source language word vectors, denotes a corresponding target language word vector, D denotes a resource dictionary.

[0032] Further, the searching of the source language synonym source language vocabulary in the source language using the fourth loss function and the inclusion of the synonym source language vocabulary in the resource dictionary vector further include the following steps:

[0033] Based on the existing semantic primitive knowledge base, a word and semantic primitive co-occurrence matrix is constructed, and the elements of the word and semantic primitive co-occurrence matrix take the value 1 to indicate that the semantic primitive X j is annotated to the source language word , otherwise it indicates that the annotation is not performed;

[0034] By decomposing the word and semantic primitive co-occurrence matrix, a fourth loss function is determined, and the semantic primitive information of the source language word vector and the resource dictionary vector are fused.

[0035] Further, the formula for fusing the semantic primitive information of the source language word vector and the resource dictionary vector is as follows:

[0036]

[0037] In the formula, X represents the set of all semantic primitives, W s S denotes the source language word, denotes W s S a corresponding source language word vector, b s and b′ j respectively denote the bias of the source language word vector W s S and the semantic primitive vector X j .

[0038] Further, the resource dictionary vector is identified by the SVM improved convolutional neural network, the related dictionary is searched, and the search results are listed, which further include the following steps:

[0039] S41, a convolutional network layer with three different filters is used, a single layer CNN convolution kernel is set in the second layer to extract three features, and a target exit technology is introduced;

[0040] S42, a LeNet5-v1 model is constructed, and it is divided into C1 layer and C2 layer;

[0041] S43, a target exit technology is introduced, an activation function, a pooling method, and a number of convolution kernels are used for Softmax regression of C1 layer and C2 layer, the neural network output is converted into a probability distribution, and is transmitted to the output layer;

[0042] S43, the output layer carries out the search of the related dictionary, and lists the search result.

[0043] Further, the C1 layer and the C2 layer are: the C2 layer obtains the feature map of the Gabor and wavelet transform extracted after the feature quantity is unified through a single-layer CNN convolution kernel, and the CNN feature map generated by the C1 layer is merged;

[0044] The C1 layer adopts PCA feature data fusion, and the feature maps of the three single features of the Gabor filter, the wavelet transform filter and the CNN convolution filter are fused to obtain a feature map.

[0045] The beneficial effects of the present application are:

[0046] 1. In the present application, the existing semantic prototype knowledge base can be reasonably utilized for cross-language vocabulary semantic prototype prediction, thereby effectively saving the manpower and time cost of semantic prototype prediction, and being beneficial to assisting language experts in semantic prototype annotation, and further faster and better constructing semantic prototype knowledge base for other languages, and having good practicability.

[0047] 2. The configuration language of different types of network equipment and the current equipment state information are unified into a single, easily identifiable, comparable and function-oriented description language, thereby unifying the description rules of the network equipment state, making it possible to quickly perform network security auditing on a large number of different types of equipment, and greatly improving the work efficiency and saving the labor cost.

[0048] 3. The present application is based on an improved convolutional neural network based on SVM, increases the spatial relationship judgment ability, separately improves the capsule network, compares the reconstruction network, optimizes the final hybrid network model, verifies the feasibility through the data set, and improves the search efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0050] Figure 1 is a flow chart of a natural source search method based on an improved convolutional neural network based on SVM according to an embodiment of the present application;

[0051] Figure 2 is a LeNet5-v1 model construction diagram in a natural source search method based on an improved convolutional neural network based on SVM according to an embodiment of the present application. DETAILED DESCRIPTION

[0052] To further illustrate the embodiments, the present application provides drawings which form a part of the present disclosure, and are included to further explain embodiments and can be used to determine the composition and operation of the various embodiments, together with the description given herein and the associated drawings. Like reference numerals have been used wherever possible to describe like elements, and implicit. drawing references made herein can be understood as a reference to like elements of the drawings.

[0053] According to an embodiment of the present application, a natural source search method based on an improved convolutional neural network of SVM is provided.

[0054] The present application will be further described by the following embodiments in conjunction with the accompanying drawings and specific implementation manners, as shown in the drawings, the natural source search method based on the improved convolutional neural network of SVM according to the embodiment of the present application includes the following steps: Figure 1

[0055] S1, collect languages of different formats, import them into the conversion program in plaintext, and generate a new language database;

[0056] In one embodiment, the collecting languages of different formats, importing them into the conversion program in plaintext, and generating a new language database further includes the following steps:

[0057] S11, scan and read the imported information piece by piece, and compare it with the existing database to confirm the function represented by the information or the current state information of the device represented by the information;

[0058] S12, mark the corresponding information function or device current state information to the newly generated database, and record the corresponding language.

[0059] S2, import the marked information in the newly generated language database into the language generation program, and analyze the feature data imported into the language generation program using the pre-built deep neural network model to automatically generate a general functional description language;

[0060] In one embodiment, the importing the marked information in the newly generated language database into the language generation program, and analyzing the feature data imported into the language generation program using the pre-built deep neural network model to automatically generate a general functional description language:

[0061] S21, map the behavior characteristics imported into the language generation program to a high-dimensional space, and use it as the input of the deep neural network model;

[0062] ​S22, training the deep neural network model parameters according to the click behavior records introduced into the language generation program as training samples;

[0063] S23, extracting the information introduced into the language generation program, calculating the correlation of the language and language conversion behavior, obtaining the correlation score between the language and language conversion behavior and sorting.

[0064] In one embodiment, the correlation of the language and language conversion behavior is calculated according to the following formula:

[0065]

[0066] In the formula, y U represents the distributed vector of the information of the language generation program extracted after analysis, y V represents the distributed vector of the language conversion behavior data set after analysis, cosine() represents the cosine function, y U T represents the transposed vector of y U , and |||| represents the vector value.

[0067] S3, randomly selecting two literal languages from the general functional description language, learning the common occurrence properties between words in each source language based on different monolingual corpora of the literal languages, and converting the properties into low-dimensional real-valued distributed representations to form resource dictionary vectors;

[0068] In one embodiment, the loss function includes a first loss function, a second loss function, a third loss function, and a fourth loss function.

[0069] The first loss function is used to learn the common occurrence properties between words in each source language, and convert the properties into low-dimensional real-valued distributed representations to generate source language word vectors.

[0070] The second loss function is used to learn the common occurrence properties between words in each target language, and convert the properties into low-dimensional real-valued distributed representations to generate target language word vectors.

[0071] The source language word vectors and the target language word vectors are used to form resource dictionary vectors.

[0072] The third loss function is determined according to the resource dictionary vector formula.

[0073] The fourth loss function is used to search for synonymous source language words in the source language, and include the synonymous source language words in the resource dictionary vectors.

[0074] In one embodiment, the resource dictionary vector formula is calculated according to the following formula:

[0075]

[0076] wherein, respectively represent the source language word and the target language word, represents the corresponding source language word vector, represents the corresponding target language word vector, and D represents the resource dictionary.

[0077] In one embodiment, the step of searching for a synonym source language word in the source language and incorporating the synonym source language word into the resource dictionary vector using the fourth loss function further comprises the following steps:

[0078] Based on the existing semantic primitive knowledge base, a word and semantic primitive co-occurrence matrix is constructed, and an element of the word and semantic primitive co-occurrence matrix is 1 indicates that the semantic primitive X j is labeled to the source language word , otherwise, it indicates that the labeling is not performed;

[0079] By decomposing the word and semantic primitive co-occurrence matrix, a fourth loss function is determined, and semantic primitive information of the source language word vector and the resource dictionary vector are fused.

[0080] In one embodiment, the formula of fusing the semantic primitive information of the source language word vector and the resource dictionary vector is as follows:

[0081]

[0082] In the formula, X represents a set of all semantic primitives, W s S represents the source language word, represents W s S the corresponding source language word vector, b s and b′ j respectively represent the source language word vector W s S and the bias of the semantic primitive vector X j .

[0083] S4, the resource dictionary vector is identified based on the SVM improved convolutional neural network, a related dictionary is searched, and the search result is listed;

[0084] In one embodiment, as shown in Figure 2 , the step of identifying the resource dictionary vector based on the SVM improved convolutional neural network, searching for a related dictionary, and listing the search result further comprises the following steps:

[0085] S41, a convolutional network layer with three different filters is adopted, a single-layer CNN convolution kernel is set in the second layer to extract three features, and a targeted dropout technique is introduced;

[0086] S42, a LeNet5-v1 model is constructed and divided into a C1 layer and a C2 layer;

[0087] S43, a targeted dropout technique is introduced, activation functions, pooling methods, and the number of convolution kernels are adopted for Softmax regression of the C1 layer and the C2 layer, the neural network output is converted into a probability distribution, and is transmitted to the output layer;

[0088] S43, the output layer searches for related dictionaries and lists the search results.

[0089] In one embodiment, the C1 layer and the C2 layer are: the C2 layer obtains the feature maps extracted by the Gabor and wavelet transforms after the dimensions are unified through a single-layer CNN convolution kernel, and the CNN feature maps generated by the C1 layer are combined; the C1 layer adopts PCA feature data fusion to fuse the feature maps of the three single features of the Gabor filter, the wavelet transform filter, and the CNN convolution filter.

[0090] In a specific application, the C1 layer and the C2 layer of the LeNet5-v1 model are divided into two parts: one is the image generated by the traditional convolutional neural network generating random convolution kernels to extract features from the input image, and the other is the image generated by the Gabor filter and the wavelet transform filter to extract features, the C2 layer obtains the feature maps extracted by the Gabor and wavelet transforms after the dimensions are unified through a single-layer CNN convolution kernel, and the CNN feature maps generated by the C1 layer are combined, the advantage is that the unimportant feature information of the feature image is simplified through a single-layer CNN, and the problems of different dimensions, different quantities, and large differences in transformation caused by different filtering and selection are unified, the C1 layer adopts PCA feature data fusion to fuse the feature maps of the three single features of the Gabor filter, the wavelet transform filter, and the CNN convolution filter. The improved convolutional neural network in the latter part mainly includes the introduction of the Targeted Dropout, the adoption of the activation function, the pooling method, and the number of convolution kernels, and finally, the fully connected layer of the LeNet5 model is replaced by Softmax regression to convert the neural network output into a probability distribution and then transmit it to the final output layer.

[0091] In addition, the classifier of the convolutional neural network is optimized by selecting the machine learning classification algorithm, comparing the random forest, GNB, SVM, and linear discrimination classification algorithm under the Caltech101 dataset, and selecting the support vector machine and the improved convolutional neural network fusion according to the experimental results. A hybrid model network model is designed and verified to be feasible. In view of the limitations of feature loss and feature information extraction in the training process of the convolutional neural network and the need for a large amount of data to train the model, the convolutional neural network model combined with multi-feature extraction and the capsule neural network are fused and designed. In order to retain the detailed information and simplify the feature dimension reduction part of the convolutional network, the generalization ability of the hybrid network model is improved by using the capsule neural network, and the spatial relationship judgment ability is increased. The advantages of the two network models are complementary. The capsule network is improved alone, and the reconstruction network is compared. The final hybrid network model is optimized and verified to be feasible through the dataset.

[0092] Random forest: Random forest is a flexible and easy-to-use machine learning algorithm that can achieve good results in most cases without hyperparameter tuning. It is one of the most commonly used algorithms, which can both classify and regress. The construction of random forest is a decision tree plus bagging method training;

[0093] GNB (Gaussian Naive Bayes): Bayesian classification is a general term for a class of classification algorithms. This classification method works based on Bayes' theorem. Naive Bayes is a classification algorithm based on probability theory. The working principle of Naive Bayes classifier is to assume that all features are independent of each other, even if the features depend on each other or depend on other features;

[0094] SVM (Support Vector Machine): Support Vector Machine (SVM) is a generalized linear classifier that classifies data in a binary manner according to supervised learning (supervised learning). The decision boundary is the maximum margin hyperplane solved by the learning sample, which has good classification and regression analysis ability;

[0095] Linear discrimination: Fisher classifier is also called Fisher linear discrimination, or linear discriminant analysis (Linear Discriminant Analysis, LDA). It originated in 1936 when Fisher linear discrimination was proposed. In 1948, it was changed to LDA linear discrimination, which has been used until now.

[0096] In specific applications, the application collects various texts (texts in Chinese, English, Korean and other languages), and simultaneously includes various text formats such as json, csv and txt.

[0097] To sum up, by means of the technical scheme of the application, the existing semantic primitive knowledge base can be reasonably utilized to predict the semantic primitives of cross-language vocabulary, so that the human and time costs of semantic primitive prediction are effectively saved, the semantic primitive annotation of language experts is facilitated, and the semantic primitive knowledge base for other languages is constructed faster and better, and the application has good practicability; the configuration languages of different types of network devices and the current device state information are unified into a single, easily identifiable, comparable and function-oriented description language, so that the description rules of the network device state are unified, and fast network security auditing of a large number of different types of devices is possible, the work efficiency is greatly improved, and the labor cost is saved.

[0098] The above only describes the preferred embodiments of the application and is not intended to limit the application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A natural source search method based on an improved convolutional neural network using SVM, characterized in that, This natural source search method includes the following steps: S1. Collect languages ​​in different formats, import them into the conversion program in plaintext, and generate a new language database; S2. Import the tagging information from the newly generated language database into the language generation program, and use a pre-built deep neural network model to analyze the feature data imported into the language generation program to automatically generate a general functional description language. S3. Randomly select two written languages ​​from the general functional description language. Based on the different monolingual corpora of the written languages, use the loss function to learn the property of co-occurrence between words in each source language, and transform this property into a low-dimensional real-valued distributed representation to form a resource dictionary vector. S4. The resource dictionary vector is identified by a convolutional neural network based on SVM, relevant dictionaries are searched, and the search results are listed. The method of using an SVM-based improved convolutional neural network to identify the resource dictionary vector, search for relevant dictionaries, and list the search results also includes the following steps: S41. A convolutional network layer with three different filters is used. A single-layer CNN convolutional kernel is set in the second layer to extract the three features, and a target exit technique is introduced. S42. Construct the LeNet5-v1 model and divide it into C1 layer and C2 layer; S43. A targeted dropout technique is introduced, which uses the activation function, pooling method, and number of convolution kernels to perform Softmax regression on layers C1 and C2, transforming the neural network output into a probability distribution and transmitting it to the output layer; the targeted dropout technique is Targeted Dropout. S44. The output layer performs a dictionary search and lists the search results.

2. The natural source search method based on an improved convolutional neural network using SVM as described in claim 1, characterized in that, The process of collecting languages ​​in different formats, importing them into the conversion program in plaintext, and generating a new language database also includes the following steps: S11. Scan and read the imported information one by one, and compare it with the existing database to confirm the function represented by the information or the current status information of the device represented by the information. S12. Mark the corresponding information function or device current status information into the newly generated database and record the corresponding language.

3. The natural source search method based on an improved convolutional neural network using SVM as described in claim 1, characterized in that, The process involves importing the labeled information from the newly generated language database into the language generation program, and using a pre-built deep neural network model to analyze the feature data imported into the language generation program to automatically generate a general functional description language. S21. Map the behavioral features imported into the language generation program into a high-dimensional space and use it as the input to the deep neural network model. S22. Train the parameters of the deep neural network model using the click behavior records imported into the language generation program as training samples; S23. Extract the information imported into the language generation program, calculate the correlation between language and language conversion behavior, obtain the correlation score between language and language conversion behavior, and rank them.

4. The natural source search method based on an improved convolutional neural network using SVM as described in claim 3, characterized in that, The formula for calculating the correlation between computational language and language conversion behavior is as follows: ； In the formula, y U y represents a distributed vector representing the information extracted from the language generation process after analysis. V This represents a distributed vector of the analyzed language conversion behavior dataset, and cosine() represents the cosine function. Indicates y U The transpose of the vector, where |||| represents the vector value.

5. The natural source search method based on an improved convolutional neural network using SVM as described in claim 1, characterized in that, The loss function includes a first loss function, a second loss function, a third loss function, and a fourth loss function; Specifically, the first loss function is used to learn the property of co-occurrence among words in each source language, and this property is transformed into a low-dimensional real-valued distributed representation to generate source language word vectors. Using the second loss function, the property of co-occurrence among target language words is learned, and this property is transformed into a low-dimensional real-valued distributed representation to generate target language word vectors; A resource dictionary vector is constructed using source language word vectors and target language word vectors; The third loss function is determined based on the resource dictionary vector formula; Using the fourth loss function, search for synonymous source language words in the source language and incorporate the synonymous source language words into the resource dictionary vector.

6. The natural source search method based on an improved convolutional neural network using SVM as described in claim 5, characterized in that, The calculation formula for the resource dictionary vector formula is as follows: ； in, , They represent source language words and target language words, respectively. express The corresponding source language word vectors, express The corresponding target language word vectors, where D represents the resource dictionary.

7. The natural source search method based on an improved convolutional neural network using SVM as described in claim 6, characterized in that, The process of using the fourth loss function to search for synonymous source language words in the source language and incorporating these synonymous source language words into the resource dictionary vector also includes the following steps: Based on the existing semantic primitive knowledge base, a word-semantic primitive co-occurrence matrix is ​​constructed, wherein the elements of the word-semantic primitive co-occurrence matrix are... A value of 1 indicates a semantic primitive. Tagged as source language words Above, otherwise it means that the aforementioned annotation has not been performed; By decomposing the word and semantic primitive co-occurrence matrix, a fourth loss function is determined, and the semantic primitive information of the source language word vectors is fused with the resource dictionary vectors.

8. The natural source search method based on an improved convolutional neural network using SVM according to claim 7, characterized in that, The formula for fusing the semantic primitives of the source language word vectors with the resource dictionary vectors is as follows: ； In the formula, X represents the set of all semantic primitives. Indicates source language words, express The corresponding source language word vectors, and Representing the word vectors of the source language respectively He Yi Yuan Vector The deviation.

9. The natural source search method based on an improved convolutional neural network using SVM as described in claim 8, characterized in that, The C1 layer and the C2 layer are as follows: The C2 layer obtains the feature map extracted by Gabor and wavelet transform after unifying the eigenscale through a single-layer CNN convolution kernel, and merges it with the CNN feature map generated by the C1 layer. Layer C1 uses PCA feature data fusion, which fuses feature maps from three sources—Gabor filter, wavelet transform filter, and CNN convolutional filter—to obtain a feature map.

Images