A patent hierarchy classification method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a patent hierarchical classification method and device, electronic equipment and storage medium. The method comprises: receiving patent text information to be classified, wherein the patent text information comprises a title and / or an abstract and / or a claim of a patent; preprocessing the patent text information to obtain preprocessed patent text information; inputting the preprocessed patent text information into a patent classification model to obtain a patent hierarchical classification result and a corresponding prediction probability value; wherein the patent classification model independently classifies the preprocessed patent text information at each level, and obtains the patent hierarchical classification result and the corresponding prediction probability value according to the result of each independent classification. The above technical solution solves the technical problems that the patent classification effect is not ideal and the accuracy of the classification result cannot be accurately predicted, and achieves the technical effects of patent automatic classification and prediction of probability value, and improvement of classification efficiency.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of deep learning, and in particular to a patent hierarchical classification method and device, electronic equipment and storage medium. BACKGROUND

[0002] The international patent classification method used in China at present has the characteristics of complex hierarchy, numerous categories and high similarity between categories, which increases the difficulty of classification.

[0003] The prior art scheme uses deep learning technology to extract features from the given patent text, and gives a prediction result based on the trained model.

[0004] However, the prior art scheme can only extract features from the given patent and then classify, and cannot accurately use the information of the patent text itself and the hierarchical information of the patent classification to classify the patent text, and the classification result is not ideal. The prior art scheme cannot meet the technical requirement of automatic patent classification and giving a prediction probability value. SUMMARY

[0005] Therefore, embodiments of the present application provide a patent hierarchical classification method and device, electronic equipment and storage medium to realize automatic patent classification and prediction of probability value, and improve the efficiency of patent classification.

[0006] In a first aspect, embodiments of the present application provide a patent hierarchical classification method, comprising:

[0007] receiving patent text information to be classified, the patent text information comprising the title and / or abstract and / or claims of the patent, the claims comprising at least one independent claim and / or at least one independent claim and at least one dependent claim of the at least one independent claim;

[0008] preprocessing the patent text information to obtain preprocessed patent text information;

[0009] inputting the preprocessed patent text information into a patent classification model to obtain patent hierarchical classification results and corresponding prediction probability values; wherein the patent classification model classifies the preprocessed patent text information at each level independently, and obtains the patent hierarchical classification results and corresponding prediction probability values according to the results of each independent classification, and the prediction probability value corresponding to the upper level classification result of the patent is greater than or equal to the prediction probability value corresponding to the lower level classification result of the patent.

[0010] In a second aspect, embodiments of the present application also provide a patent hierarchical classification device, which comprises:

[0011] The patent text information receiving module is configured to receive patent text information to be classified, wherein the patent text information comprises a title and / or an abstract and / or a claim of a patent.

[0012] The preprocessing module is configured to preprocess the patent text information to obtain preprocessed patent text information.

[0013] The classification module is configured to input the preprocessed patent text information into a patent classification model to obtain patent classification results at each level and corresponding prediction probability values, wherein the patent classification model independently classifies the preprocessed patent text information at each level, and obtains the patent classification results at each level and the corresponding prediction probability values according to the classification results at each level, wherein the corresponding prediction probability value of a higher-level classification result of a patent is greater than or equal to the corresponding prediction probability value of a lower-level classification result of the patent.

[0014] In a third aspect, an electronic device is provided, and the electronic device comprises:

[0015] one or more processors;

[0016] a storage device configured to store one or more programs,

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the patent hierarchical classification method provided by any of the embodiments of the present application.

[0018] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the patent hierarchical classification method provided by any of the embodiments of the present application.

[0019] The technical solution of the embodiments of the present application receives patent text information to be classified, wherein the patent text information comprises a title and / or an abstract and / or a claim of a patent; preprocesses the patent text information to obtain preprocessed patent text information; inputs the preprocessed patent text information into a patent classification model to obtain patent classification results at each level and corresponding prediction probability values; wherein the patent classification model independently classifies the preprocessed patent text information at each level, and obtains the patent classification results at each level and the corresponding prediction probability values according to the classification results at each level, wherein the corresponding prediction probability value of a higher-level classification result of a patent is greater than or equal to the corresponding prediction probability value of a lower-level classification result of the patent, thereby solving the technical problems of unsatisfactory patent classification results and the inability to accurately predict the accuracy of the classification results, achieving patent automatic classification and prediction of probability values, and improving the classification efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the example embodiments of the present application, the following briefly introduces the drawings required in the description of the embodiments. Obviously, the drawings described are only a part of the drawings of the present application, and not all the drawings. For those skilled in the field, other drawings can also be obtained from these drawings without creative labor.

[0021] Figure 1 is a flowchart of a patent classification method provided by the first embodiment of the present application;

[0022] Figure 2 is a Skip-gram model diagram provided by the first embodiment of the present application;

[0023] Figure 3 is a local deep network diagram provided by the first embodiment of the present application;

[0024] Figure 4 is a patent classification model diagram provided by the first embodiment of the present application;

[0025] Figure 5 is a structural diagram of a patent classification device provided by the second embodiment of the present application;

[0026] Figure 6 is a structural diagram of an electronic device provided by the third embodiment of the present application. DETAILED DESCRIPTION

[0027] The present application will be further described below in conjunction with the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present application, and not to limit the present application. In addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings, not all the structures.

[0028] In addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings, not all the contents. Before discussing the example embodiments in more detail, it should be mentioned that some example embodiments are described as processes or methods depicted by flowcharts. Although the flowchart describes each operation (or step) as a sequential process, many of the operations can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operations are completed, but can also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, etc.

[0029] Embodiment one

[0030] Figure 1A flowchart of a patent classification method provided by the first embodiment of the present application is shown in the figure. The embodiment can be applied to the classification of patent text. The method can be executed by a patent classification device. The device can be implemented by software and / or hardware. The device can be configured in a terminal and / or a server to implement the patent classification method of the present application.

[0031] As shown in the figure, the method of the embodiment can specifically include the following steps. Figure 1

[0032] S110, receiving patent text information to be classified, wherein the patent text information includes the title and / or abstract and / or claims of the patent.

[0033] The patent text information can be part of the information in a patent standard file. The patent standard file can include the abstract of the specification, the abstract drawing, the claims, the specification, and the specification drawing. The patent text information includes the title and / or abstract and / or claims of the patent. For example, the patent text information includes the title and abstract of the patent. The patent text information can also include the title and claims of the patent.

[0034] S120, preprocessing the patent text information to obtain preprocessed patent text information.

[0035] The preprocessing can be tokenization, stop word removal, and other processing of patent data. The preprocessing technology is used to clean and tokenize the patent text. The preprocessed patent text is helpful for feature extraction and the improvement of the performance of the classification algorithm.

[0036] The embodiment is to preprocess the patent text. The patent text is a special text, which is different from common texts. There are many domain-specific words. The main source of the low accuracy of the patent text tokenization is the recognition of patent terms. The embodiment uses a patent term dictionary collected and sorted to alleviate the low accuracy of the tokenization and solve the problem of the recognition of unregistered words.

[0037] ​After the patent text is segmented, the segmented results include not only patent terms but also some feature words that are useless for classification. In this embodiment, these words are regarded as useless words or general words. For example, "the invention", "disclosed", "a kind of", "relates to", "based on", and "method" in the patent text. These general words will appear in the patent text basically and have little contribution to classification, and even can reduce the accuracy. There are also various symbols such as ",", "。", and "+" and the like, which are useless for classification. Therefore, this embodiment adds specific stop words of the patent text to the common stop list to filter the stop words, and combines the general words to remove various symbols and words that have no meaning. After the patent text is segmented, the segmented results and the stop word list are traversed respectively. If the segmented results contain symbols and words in the stop word list, they are removed to obtain the preprocessed patent text information.

[0038] In S130, the preprocessed patent text information is input into the patent classification model to obtain the patent classification results of each level and the corresponding prediction probability values. The patent classification model independently classifies the preprocessed patent text information at each level, and obtains the patent classification results of each level and the corresponding prediction probability values according to the classification results of each level. The corresponding prediction probability value of the upper level classification result of the patent is greater than or equal to the corresponding prediction probability value of the lower level classification result of the patent.

[0039] The patent classification model can be a neural network model based on deep learning. Through the patent classification model, the patent classification results of each level and the corresponding prediction probability values can be obtained. The patent levels can be levels classified according to the International Patent Classification (IPC), such as five levels of department, large category, small category, large group, and small group, or different levels classified according to other standards. The obtained patent classification results of each level can be the classification results corresponding to each level. For example, if classified according to the IPC, the classification results of the patent text information at the department level and at the large category, small category, large group, and small group levels can be obtained. The obtained patent classification results of each level and the corresponding prediction probability values can be that each level classification result corresponds to a prediction probability value. The probability value is the correct probability of the corresponding classification result. For example, a patent text information is classified. At the department level, the classification result is H department, and the corresponding prediction probability value is 80%. At the large category level, the classification result is H01, and the corresponding prediction probability value is 70%.

[0040] It can be understood that the patent classification model independently classifies each level of the preprocessed patent text information, and obtains the classification results of each level of the patent and the corresponding prediction probability value according to the results of each independent classification. The prediction probability value corresponding to the upper level classification result of the patent is greater than or equal to the prediction probability value corresponding to the lower level classification result of the patent. Taking the IPC patent classification as an example, the patent classification model independently classifies the five levels of the preprocessed patent text information, that is, the H department level, the G01 class level, the H01 group level, the H01 group level, and the H01 group level. The technical solution of the present embodiment is that the patent classification model independently classifies each level of the preprocessed patent text information, and obtains the classification results of each level of the patent and the corresponding prediction probability value according to the results of each independent classification. The prediction probability value can be obtained according to a certain algorithm after obtaining the classification results of each level of the patent. For example, if the classification result conforms to the five levels of the IPC, the contradictory classification result means that the classification results of two levels of the patent exclude each other. For example, in the department level, the classification result is H department, and in the class level, the classification result is G01. At least one of the two classification results is incorrect, and there is no correct condition.

[0041] The non-contradictory classification result means that the classification results of two levels of the patent do not exclude each other. For example, the department level is classified into H department, and the class level is classified into H01. In this case, there is a probability that both the classification results of the two levels are correct. Whether the classification results of more than three patents are contradictory can be compared two by two. When the classification results do not exclude each other, all sub-models output their own classification results, which correspond to the classification results of each level. The other four level sub-models play a role of mutual verification in this case. The prediction probability value corresponding to the class level is set to 90%, which is not limited and can be adjusted in actual classification. It is sufficient that the prediction probability value corresponding to the upper level classification result of the patent is greater than or equal to the prediction probability value corresponding to the lower level classification result of the patent. For example, in the model training process, the correct frequency value of each level of the sample is used as the classification result of each level and the corresponding prediction probability value, or the correct frequency value of the sample and the characteristics of the classification task are combined to set the classification result of each level and the corresponding prediction probability value. The purpose of setting the classification result of each level and the corresponding prediction probability value is to represent the accuracy of each level of classification. If the accuracy of the group level is 100%, the classification result of the group level can be directly determined. The accuracy of the classification result of the group level cannot be lower than 100%. Therefore, the prediction probability value corresponding to the upper level classification result of the patent is greater than or equal to the prediction probability value corresponding to the lower level classification result of the patent, which is consistent with the actual situation. The more detailed the classification is, the greater the probability of error is, and the lower the accuracy is.

[0042] For example, the part level is classified into H, the large class level is classified into H01, the small class level is classified into H01B, the large group level is classified into H01B1 / 00, and the small group level is classified into H01B1 / 02, a higher classification accuracy can be determined, for example, the prediction probability value corresponding to the part level is 95%, of course, as the patent level is refined, the classification result may deviate, the prediction probability value corresponding to the lower level classification result is lower than the prediction probability value corresponding to the patent upper level classification result, the prediction probability value corresponding to the large class level is set to 90%, the prediction probability value corresponding to the small class level is set to 90%, the prediction probability value corresponding to the large group level is set to 85%, and the prediction probability value corresponding to the small group level is set to 80%, the set probability value should satisfy that the prediction probability value corresponding to the patent upper level classification result is greater than or equal to the prediction probability value corresponding to the patent lower level classification result, in the case that the classification results do not contradict each other, the probability values set for each level can be equal, or determined according to the actual classification, which is not limited here.

[0043] If the classification results at each level contradict each other, the number of the largest non-contradictory classification results can be found, the number of the largest non-contradictory classification results can be determined by the number of classification results at each level in the non-contradictory group, the classification results at each level in the non-contradictory group do not contradict each other, and the classification results in the non-contradictory group contradict each other, thus, the largest non-contradictory classification result can be the number of classification results in the non-contradictory group at each level, for example, the part level is classified into H, the large class level is classified into H01, the small class level is classified into H01B, the large group level is classified into G01B3 / 00, and the small group level is classified into G01B3 / 02, at this time, the non-contradictory group can be two groups, the first group is that the part level is classified into H, the large class level is classified into H01, and the small class level is classified into H01B, the second group is that the large group level is classified into G01B3 / 00, and the small group level is classified into G01B3 / 02, the classification results in the two groups are 3 and 2 respectively, and the number of the largest non-contradictory classification result is 3. The method of finding the non-contradictory group can be to compare the classification results at each level two by two, divide the non-contradictory into the same group, and then compare whether the remaining classification results at each level contradict the group, if not, they are included in the group, and finally at least one non-contradictory group is formed, and the number of the largest non-contradictory classification result is obtained. The patent classification results at each level and the corresponding prediction probability value are determined according to the number of the largest non-contradictory classification result, the prediction probability value can be related to the number of the largest non-contradictory classification result, for example, the number of the largest non-contradictory classification result is 4, which indicates that only one level of classification result may be wrong, and the accuracy of the remaining classification results is relatively high, if the number of the largest non-contradictory classification result is 1, it indicates that the classification result is completely chaotic, and the overall patent text classification result is not good, and the classification results at each level and the corresponding prediction probability value should be low. The purpose of modifying the upper level classification result of the largest non-contradictory classification result is to avoid obvious classification errors.

[0044] For example, if the number of maximum non-contradictory classification results is 4, then the classification results and corresponding predicted probability values ​​for each level are set to 80%. Alternatively, the predicted probability values ​​can be decreased progressively with each level. The larger the number of maximum non-contradictory classification results, the higher the classification results and corresponding predicted probability values ​​for each level within each maximum non-contradictory group. For instance, if the department level is classified as H, the major category level as H01, the minor category level as H01B, the major group level as H01B3 / 00, and the minor group level as G01B3 / 02, the predicted probability value for the department level can be set to 80%, the major category level as 80%, the minor category level as 70%, the major group level as 70%, and the minor group level as 10%. If the classification result of the upper level of the most consistent classification result contradicts it, the upper level classification result of the most consistent classification result can be modified, and its corresponding predicted probability value can be determined. For example, if the classification result is: department level is classified as G, major category level as H01, minor category level as H01B, major group level as H01B1 / 00, and minor group level as H01B1 / 02, then except for the department level which is classified as G, all other levels are classified as H, and the classification results of other levels are not contradictory. In this case, the department level classification result can be modified to H to correct possible classification errors. The terms "upper level" and "lower level" are relative. For example, major category, major group, and department: major category is the upper level of major group, major category is the lower level of department, and major group is the lower level of major category.

[0045] Optionally, based on any of the optional technical solutions in the embodiments of the present invention, the patent classification model includes multiple independent classification sub-models, each independent classification sub-model corresponding to a patent classification level, and each independent classification sub-model classifies the preprocessed patent text information to obtain the classification result corresponding to the patent classification level of the independent classification sub-model.

[0046] The independent classification sub-models are parameter-independent independent classification sub-models, and the plurality of independent classification sub-models can be the same model or different models, and the parameters of the same models are independent. The independent classification sub-models correspond to the levels of the patent classification, which can be understood as that the patent classification model comprises a plurality of independent classification sub-models, and each independent classification sub-model corresponds to a level of the patent classification. For example, the patent classification model can comprise five independent classification sub-models, namely, a first independent classification sub-model, a second independent classification sub-model, and a fifth independent classification sub-model. The first independent classification sub-model corresponds to the section level, the second independent classification sub-model corresponds to the class level, and the fifth independent classification sub-model corresponds to the subgroup level. The independent classification sub-models classify the preprocessed patent text information to obtain the classification results of the levels of the patent classification corresponding to the independent classification sub-models. It can be understood that the first independent classification sub-model classifies the preprocessed patent text information to obtain the classification results of the section level, such as section H, the second independent classification sub-model classifies the preprocessed patent text information to obtain the classification results of the class level, such as class H01, and so on.

[0047] Optionally, on the basis of any optional technical solution in the embodiments of the present application, the plurality of independent classification sub-models are the same classification sub-models or different classification sub-models set according to the characteristics of different levels of the patent.

[0048] It can be understood that even if the plurality of independent classification sub-models are the same classification sub-models, the parameters of each independent classification sub-model are independent. The different classification sub-models set according to the characteristics of different levels of the patent can be different classification sub-models set according to different levels, such as the section level and the subgroup level. The range of the section level is much larger than that of the subgroup level, and the requirements for the independent classification sub-models can also be different. At this time, different independent classification sub-models can be set according to different task scenarios to cope with different classification tasks, so as to achieve better classification effect.

[0049] Optionally, on the basis of any optional technical solution in the embodiments of the present application, the independent classification sub-models comprise an embedding layer, a key feature layer, a local deep network layer, a global feature layer, and a classification layer. The embedding layer obtains a word vector according to the preprocessed patent text information. The key feature layer extracts a key feature vector from the word vector. The local deep network layer extracts a local key feature vector from the key feature vector. The global feature layer extracts a global key feature vector from the key feature vector. The classification layer fuses the local key feature vector and the global key feature vector to obtain a fused feature vector, and classifies according to the fused feature vector.

[0050] The embedding layer can embed text into a mathematical space, enabling the computer to recognize the data type. After text preprocessing, the text needs to be converted into a numerical value or a vector, so that the computer can understand and process the text. In the process of vectorization, the original spatial information should be preserved as much as possible. The loss of information in spatial mapping will have an important impact on the accuracy of subsequent model classification. Therefore, the representation of the text vector is crucial. The method for representing text in the embodiment can be divided into three types, namely discrete representation method, distributed representation method and dynamic representation method.

[0051] The representative of the discrete representation method is the bag-of-words model with high dimension, sparsity and discreteness, such as One-hot and N-gram model. All the words or characters in the corpus are summarized to obtain K words or characters, i.e. a word or character dictionary with a length of K. Then, the one-hot representation is to encode each word or character with an index. The dimension of each word or character code is K, and each position corresponds to a word or character. If the word or character in the represented text appears in the dictionary, it is marked as 1 or its frequency of appearance, otherwise it is marked as 0 in its corresponding position. For the representation of text, the vectors of each word or character in the text are added.

[0052] The distributed representation has Word2vec, which mainly includes CBOW and Skip-gram two learning methods. Both of them need to set a window value in advance. In the window, the first way is to predict a word through the front and back information, and the second way is to predict the front and back information through the current word. The second learning method is adopted in the embodiment, such as Figure 2As shown, the model of the second learning mode is composed of an input layer, a hidden layer and an output layer. In the process of realizing the word vector, the negative sampling technique is used to improve the training speed and train the model parameters, and finally the obtained parameter matrix is the word vector. The Word2vec network structure is simple and effective, and the training speed is fast. Although only the context information in a limited range is considered, the training corpus is sufficient, and the effect of the produced word vector is also very good. In this embodiment, the training corpus of the word vector adopts a comprehensive large corpus, including Chinese encyclopedia data, Chinese Wikipedia data, etc., so that the word vector model can learn the semantic and grammatical relationship in the sentence to obtain a more meaningful word vector.

[0053] In the dynamic representation method, BERT is the most semantic representative word vector, which solves the problem of polysemy by learning context information, inter-sentence logical information and order information.

[0054] The classification layer fuses the local key feature vector and the global key feature vector to obtain a fusion feature vector, and classifies according to the fusion feature vector. It can be understood that: first, the output vector of the local deep network layer and the output vector of the global feature layer are fused and sent to the full connection layer. If the patent text of each level includes R categories, the output of the fusion layer can be converted to an R-dimensional vector through the full connection layer. Then the output vector of the full connection layer is sent to the softmax layer to estimate the classification number to which the to-be-tested patent text belongs by using the softmax function. The formula of the softmax function is as follows:

[0055]

[0056] In the formula, R is the total number of categories in the layer, is the score of the i-th sample in the j-th category in the layer, and represents the category probability of sample i classified into the j-th category in the layer. Through the formula, the probability of sample i in each category of each layer can be obtained, and the category with the highest probability score is taken as the final classification number of sample i in the layer.

[0057] For the multi-level classification problem of patent text, the loss function of the present application is the mainstream cross-entropy loss function, and its specific formula is as follows:

[0058]

[0059] In the formula, e is the number of samples, R is the total number of categories, represents the probability that the i-th sample is actually in the j-th category in the layer, represents the probability that sample i is classified into the j-th category in the layer.

[0060] In order to make the loss function converge faster, find the relative minimum loss value, the embodiment uses the Adam optimizer. On the basis of using the adaptive Adam optimizer, a learning rate decay strategy is used at the same time. The strategy is the ReduceLROnPlateau decay method, which can dynamically realize the reduction of the learning rate according to the results of the validation set. The parameters set in the embodiment are that the learning rate is reduced by 90% when the loss value of the validation set does not decrease for two consecutive iterations, so that the model achieves relatively good training effect.

[0061] Optionally, on the basis of any optional technical solution in the embodiment of the application, the key feature layer is a double-layer attention mechanism layer.

[0062] It can be understood that each word has different contribution to the classification of the patent text category, and the attention mechanism can assign different weights to each word to represent the contribution of each word. The double-layer attention mechanism layer can be an attention mechanism for relationship classification, which gives different weights to words in a sentence when outputting the LSTM vector, verifying the effectiveness of the attention mechanism in the text task. In order to give more weight to the words that contribute most to the classification of the patent text, improve the understanding of the whole text; at the same time, in order to reduce the loss of key information in the process of deep learning of the model, the attention mechanism is introduced into the multi-feature fusion model, and the calculation formula is as follows:

[0063]

[0064] Wherein, d is the output vector of the embedding layer, the dimension is n v, n is the number of words, v is the dimension of a word vector, more abstract features are learned through the tanh function, the output is the score between different word vectors, W represents the training parameter, the dimension is v, represents the transpose W T , The normalized weight value obtained by M through the softmax function is the weight value, which represents the correlation between different word vectors, and finally the weight is applied to the word vector for weighting and summing to obtain the value More key information can be highlighted.

[0065] Optionally, on the basis of any optional technical solution in the embodiment of the application, the local deep network layer comprises:

[0066] Three different parallel convolution layers; they are respectively a convolution kernel size of 2 n, the number of convolution kernels is 256, the convolution kernel size is 3 n, the number of convolution kernels is 256, the convolution kernel size is 4 n, the number of convolution kernels is 256; wherein, n is the dimension of the key feature vector;

[0067] Correspondingly, the local deep network layer performs local key feature extraction on the key feature vector, including:

[0068] with a convolution kernel size of 2 n, a convolution kernel number of 500, a convolution kernel size of 3 n, a convolution kernel number of 500, a convolution kernel size of 4 n, a convolution kernel number of 500, respectively, to obtain bi-gram feature vectors, tri-gram feature vectors, and four-gram feature vectors;

[0069] with a kernel size of and a stride of The bi-gram feature vectors, tri-gram feature vectors, and four-gram feature vectors are respectively pooled by a max-pooling layer with a kernel size of

[0070] The bi-gram pooling feature vectors, tri-gram pooling feature vectors, and four-gram pooling feature vectors are respectively subjected to two layers of equal-length convolution, each layer of equal-length convolution adopts a residual connection structure, and an activation function is executed in advance before each equal-length convolution layer is executed, and the convolution kernel size of the two layers of equal-length convolution is 3 1, a convolution kernel number of 500.

[0071] It can be understood that the local deep network layer mainly extracts features by using CNN and deep network. In order to better obtain key local features in the patent text, the output of the key feature layer is taken as the input of the local deep network layer, so that the CNN can extract more important n-gram features.

[0072] In the local deep network layer, there are three different parallel convolution layers. They are respectively a convolution kernel size of 2 n, a convolution kernel number of 500; a convolution kernel size of 3 n, a convolution kernel number of 500; a convolution kernel size of 4 n, a convolution kernel number of 500. Bi-gram features, tri-gram features, and four-gram features can be extracted respectively, so that the model can learn multiple types of local area features that can complement each other. Among them, n is the dimension of the key feature vector, and if the key feature vector is a vector group, n is the dimension of a key feature vector in the vector group.

[0073] Taking tri-gram features as an example, after extracting the tri-gram features, subsequent steps are performed to extract long-distance dependency features. For example... Figure 3 As shown on the left, firstly, after passing through a convolution kernel of size 3... 1 represents a max-pooling layer with a stride of 2. This downsampling with a stride of 2 halves the size of each patent text representation. After downsampling, the computation time or data size of subsequent deep networks is halved. If downsampling is performed L times, it can represent lengths of 2... L Related words within the range. In this embodiment, L is preferably 1.

[0074] Then, after two convolutional layers of equal length, each with a kernel size of 3... 1. The number of convolutional kernels is 500; the kernel size is 3. 1. A convolutional layer with 500 kernels.

[0075] To more vividly explain the changes in word positions in deep networks, such as Figure 3 As shown on the right, the top layer is the output of the tri-gram features, assuming 8 word positions. The middle layer is the output after a max-pooling layer with a stride of 2, reducing it to 4 word positions. The last layer is the output after an equal-length convolutional layer, still containing 4 word positions. The features at each word position can be understood as a compression of the contextual information of the corresponding word position in the previous layer sequence and the one word position above and below it. In other words, the features at each word position in each layer sequence are more accurate semantics modified by contextual information. With two equal-length convolutional layers and the training of a deep network, the features at each word position will contain more contextual information, resulting in richer semantics. Equal-length convolutions are used because they ensure that the input and output lengths are consistent, resulting in the same number of word positions.

[0076] Finally, to support the training of deep networks, residual connections are used, connecting the output of the downsampling layer to the output of the next convolutional layer each time, which alleviates the gradient vanishing problem. Furthermore, an activation function, the Rectified Linear Unit (ReLU), is pre-executed before each equal-length convolutional layer. ReLU, as a non-saturating activation function, also alleviates the gradient vanishing problem and converges faster. More importantly, it allows the network to introduce sparsity, improving training speed. In addition, to enable fusion with bi-gram and four-gram features, padding is used after convolutional layers in deep networks.

[0077] Optionally, in the embodiment of the present application, on the basis of any optional technical solution, the global feature layer comprises a bidirectional long short-term memory neural network; the global feature layer extracts global key features from the key feature vector to obtain a global key feature vector, and the global key feature vector comprises:

[0078] The key feature vector is extracted by the bidirectional long short-term memory neural network to obtain an extracted vector.

[0079] The extracted vector and the key feature vector are spliced to obtain a spliced vector.

[0080] The spliced vector is subjected to maximum pooling to obtain a global key feature vector.

[0081] It can be understood that, in order to better obtain key context features of the patent text, the output of the key feature layer is also taken as the input of the global feature layer. The bidirectional long short-term memory neural network is a combination of two opposite direction LSTMs, so that the network has forward and backward information of the sequence at each time, thereby better understanding or extracting context features. And the gradient vanishing problem is alleviated to a certain extent through the input gate, the output gate and the forget gate. And the output of the word embedding is spliced with the output of the BiLSTM network to extract more rich global features. Since the patent text classification is sensitive to some keywords or terms, therefore, a maximum pooling layer with a step of 1 is performed after the output of the BiLSTM network to extract key features in the global features.

[0082] As shown in Figure 4 , it is a patent classification model schematic diagram of the embodiment, from bottom to top in turn is embedding layer, key feature layer, local deep network layer, global feature layer and classification layer; the embedding layer obtains a word vector according to the preprocessed patent text information; the key feature layer extracts features from the word vector to obtain a key feature vector; the local deep network layer extracts local key features from the key feature vector to obtain a local key feature vector; the global feature layer extracts global key features from the key feature vector to obtain a global key feature vector; the classification layer fuses the local key feature vector and the global key feature vector to obtain a fused feature vector, and classifies according to the fused feature vector.

[0083] The technical scheme of the embodiment receives patent text information to be classified, the patent text information including a title and / or an abstract and / or a claim of a patent; pre-processes the patent text information to obtain pre-processed patent text information; inputs the pre-processed patent text information into a patent classification model to obtain patent classification results of each level and corresponding prediction probability values; wherein the patent classification model independently classifies the pre-processed patent text information at each level, and obtains the patent classification results of each level and the corresponding prediction probability values according to the results of each independent classification, the corresponding prediction probability value of the upper level classification result of the patent being greater than or equal to the corresponding prediction probability value of the lower level classification result of the patent, thereby solving the technical problems of unsatisfactory patent classification effect and inability to accurately predict the accuracy of the classification results, achieving automatic patent classification and prediction of probability values, and improving the classification efficiency.

[0084] Embodiment Two

[0085] Figure 5 A structure diagram of a patent hierarchical classification device provided for the second embodiment of the application, the patent hierarchical classification device provided by the embodiment can be implemented by software and / or hardware, and can be configured in a terminal and / or a server to implement the patent hierarchical classification method in the embodiment. The device can specifically include: a patent text information receiving module 210, a pre-processing module 220, and a classification module 230.

[0086] The patent text information receiving module 210 is configured to receive patent text information to be classified, the patent text information including a title and / or an abstract and / or a claim of a patent; the pre-processing module 220 is configured to pre-process the patent text information to obtain pre-processed patent text information; and the classification module 230 is configured to input the pre-processed patent text information into a patent classification model to obtain patent classification results of each level and corresponding prediction probability values; wherein the patent classification model independently classifies the pre-processed patent text information at each level, and obtains the patent classification results of each level and the corresponding prediction probability values according to the results of each independent classification, the corresponding prediction probability value of the upper level classification result of the patent being greater than or equal to the corresponding prediction probability value of the lower level classification result of the patent.

[0087] The technical scheme of the embodiment is as follows: patent text information to be classified is received, the patent text information including a title and / or an abstract and / or a claim of a patent; the patent text information is preprocessed to obtain preprocessed patent text information; the preprocessed patent text information is input into a patent classification model to obtain patent classification results at each level and corresponding prediction probability values; the patent classification model performs independent classification at each level on the preprocessed patent text information, and obtains the patent classification results at each level and the corresponding prediction probability values according to the results of the independent classification at each level, and the corresponding prediction probability value of an upper level classification result of the patent is greater than or equal to the corresponding prediction probability value of a lower level classification result of the patent, thereby solving the technical problems of an unsatisfactory patent classification effect and an inability to accurately predict the accuracy of the classification results, and achieving the technical effects of automatic patent classification and prediction of probability values and improved classification efficiency.

[0088] On the basis of any optional technical scheme in the embodiments of the application, optionally, the patent classification model comprises a plurality of independent classification sub-models, the independent classification sub-models corresponding to the levels of patent classification, and the independent classification sub-models performing classification on the preprocessed patent text information to obtain classification results at the levels of patent classification corresponding to the independent classification sub-models.

[0089] On the basis of any optional technical scheme in the embodiments of the application, optionally, the plurality of independent classification sub-models are the same classification sub-models or different classification sub-models set according to the characteristics of different levels of patents.

[0090] On the basis of any optional technical scheme in the embodiments of the application, optionally, the independent classification sub-models comprise an embedding layer, a key feature layer, a local deep network layer, a global feature layer, and a classification layer; the embedding layer obtains a word vector according to the preprocessed patent text information; the key feature layer performs feature extraction on the word vector to obtain a key feature vector; the local deep network layer performs local key feature extraction on the key feature vector to obtain a local key feature vector; the global feature layer extracts global key features from the key feature vector to obtain a global key feature vector; and the classification layer fuses the local key feature vector and the global key feature vector to obtain a fused feature vector and performs classification according to the fused feature vector.

[0091] On the basis of any optional technical scheme in the embodiments of the application, optionally, the local deep network layer comprises:

[0092] three different parallel convolution layers; they are a convolution kernel size of 2 n, a number of convolution kernels of 500, a convolution kernel size of 3 n, a number of convolution kernels of 500, and a convolution kernel size of 4 n, a number of convolution kernels of 500; wherein n is the dimension of the key feature vector.

[0093] Correspondingly, the local deep network layer performs local key feature extraction on the key feature vector, including:

[0094] using a convolution kernel size of 2 n, a convolution kernel quantity of 500, a convolution kernel size of 3 n, a convolution kernel quantity of 500, a convolution kernel size of 4 n, a convolution kernel quantity of 500, respectively performing convolution on the key feature vector to obtain a bi-gram feature vector, a tri-gram feature vector and a four-gram feature vector;

[0095] using a kernel size of 3 1 and a stride of 2 to perform maximum pooling on the bi-gram feature vector, the tri-gram feature vector and the four-gram feature vector respectively, and repeatedly performing L times to obtain a bi-gram pooling feature vector, a tri-gram pooling feature vector and a four-gram pooling feature vector, wherein L is a positive integer greater than or equal to 1;

[0096] respectively performing two equal-length convolution layers on the bi-gram pooling feature vector, the tri-gram pooling feature vector and the four-gram pooling feature vector, each equal-length convolution layer adopting a residual connection structure, and performing an activation function in advance before each equal-length convolution layer is executed, and the convolution kernel size of the two equal-length convolution layers is 3 1, and the convolution kernel quantity is 500.

[0097] In the embodiment of the application, optionally, the global feature layer comprises a bidirectional long short-term memory neural network; the global feature layer extracts global key features from the key feature vector to obtain a global key feature vector, including:

[0098] extracting the key feature vector through the bidirectional long short-term memory neural network to obtain an extracted vector;

[0099] splicing the extracted vector and the key feature vector to obtain a spliced vector;

[0100] performing maximum pooling on the spliced vector to obtain a global key feature vector.

[0101] In the embodiment of the application, optionally, the key feature layer is a double-layer attention mechanism layer.

[0102] The above patent classification device can execute the patent classification method provided by any embodiment of the application, and has corresponding functional modules and beneficial effects of executing the patent classification method.

[0103] Example 3

[0104] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention, as shown below. Figure 6 As shown, the electronic device includes a processor 310, a memory 320, an input device 330, and an output device 340; the number of processors 310 in the device can be one or more. Figure 6 Taking a processor 310 as an example; the processor 310, memory 320, input device 330, and output device 340 in the device can be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.

[0105] The memory 320, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the patent hierarchy classification method in this embodiment of the invention. The processor 310 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 320.

[0106] The memory 320 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on terminal usage. Furthermore, the memory 320 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory, or other non-volatile solid-state storage device. In some instances, the memory 320 may further include memory remotely located relative to the processor 310, which can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0107] Input device 330 can be used to receive input digital or character information, and to generate signal inputs related to user settings and function control of the device. Output device 340 may include display devices such as a display screen.

[0108] Example 4

[0109] The fourth embodiment of the present application also provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to perform a patent hierarchical classification method, the method comprising: receiving patent text information to be classified, the patent text information comprising a title and / or an abstract and / or a claim of a patent; pre-processing the patent text information to obtain pre-processed patent text information; inputting the pre-processed patent text information into a patent classification model to obtain patent hierarchical classification results and corresponding prediction probability values; wherein the patent classification model independently classifies the pre-processed patent text information at each level, and obtains the patent hierarchical classification results and corresponding prediction probability values according to the results of each independent classification, and the prediction probability value corresponding to an upper level patent classification result is greater than or equal to the prediction probability value corresponding to a lower level patent classification result.

[0110] The computer storage medium of the embodiments of the present application can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may, for example, be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples (non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0111] The computer readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which computer readable program code is embodied. Such propagated data signal can take multiple forms, including but not limited to electromagnetic signal, optical signal or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in connection with an instruction execution system, apparatus or device.

[0112] The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination thereof.

[0113] Computer program code for carrying out operations of embodiments of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0114] It is to be understood that the above description is merely a preferred embodiment of the application and the applied technical principles. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, modifications and substitutions can be made without departing from the scope of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A patent hierarchy classification method, characterized by, The method comprises the following steps: receiving patent text information to be classified, wherein the patent text information comprises a title and / or an abstract and / or claims of a patent, the claims comprising at least one independent claim and / or at least one independent claim and at least one dependent claim of the at least one independent claim; preprocessing the patent text information to obtain preprocessed patent text information; inputting the preprocessed patent text information into a patent classification model to obtain patent classification results at each level and corresponding prediction probability values; wherein the patent classification model performs independent classification at each level on the preprocessed patent text information, and obtains the patent classification results at each level and the corresponding prediction probability values according to the results of the independent classification at each level, wherein the corresponding prediction probability value of the upper level classification result of the patent is greater than or equal to the corresponding prediction probability value of the lower level classification result of the patent; the patent classification model comprises a plurality of independent classification sub-models, wherein the independent classification sub-models correspond to the levels of patent classification, and the independent classification sub-models perform classification on the preprocessed patent text information to obtain the classification results of the levels of patent classification corresponding to the independent classification sub-models; the independent classification sub-models comprise an embedding layer, a key feature layer, a local deep network layer, a global feature layer, and a classification layer; the embedding layer obtains a word vector according to the preprocessed patent text information; the key feature layer extracts key features from the word vector to obtain a key feature vector; the local deep network layer extracts local key features from the key feature vector to obtain a local key feature vector; the global feature layer extracts global key features from the key feature vector to obtain a global key feature vector; and the classification layer fuses the local key feature vector and the global key feature vector to obtain a fused feature vector, and performs classification according to the fused feature vector; the local deep network layer comprises: three different parallel convolution layers; they are respectively a first convolution layer with a convolution kernel size of 2 n, a second convolution layer with a convolution kernel size of 3 n, a second convolution layer with a convolution kernel size of 4 n, a third convolution layer with a convolution kernel size of 4; wherein n is a dimension of the key feature vector; correspondingly, the local deep network layer extracts local key features from the key feature vector, which comprises: with a kernel size of 2 n, a first convolutional layer with 500 kernels of size 3 n, a second convolutional layer with 500 kernels of size 4 n, a third convolutional layer with 500 kernels, respectively convolving the key feature vectors to obtain bi-gram feature vectors, tri-gram feature vectors and four-gram feature vectors.

2. The method of claim 1, wherein, the plurality of independent classification sub-models are the same classification sub-models, or different classification sub-models set according to the characteristics of different levels of patents.

3. The method of claim 1, wherein, with a kernel size of 3 1and a stride of 2, and is repeated L times to obtain bi-gram pooling feature vectors, tri-gram pooling feature vectors, and four-gram pooling feature vectors, where L is a positive integer greater than or equal to 1. Two layers of equal-length convolution are respectively performed on the bi-gram pooling feature vector, the tri-gram pooling feature vector and the four-gram pooling feature vector, each layer of equal-length convolution adopts a residual connection structure, and an activation function is performed in advance before each layer of equal-length convolution is performed, and the convolution kernel size of the two layers of equal-length convolution is 3 1, and the number of convolution kernels is 500.

4. The method of claim 1, wherein, the global feature layer comprises a bidirectional long short-term memory neural network; and the global feature layer extracts global key features from the key feature vector to obtain a global key feature vector, which comprises: extracting the key feature vector through the bidirectional long short-term memory neural network to obtain an extracted vector; splicing the extracted vector and the key feature vector to obtain a spliced vector; performing maximum pooling on the spliced vector to obtain a global key feature vector.

5. The method of claim 1, wherein, the key feature layer is a double-layer attention mechanism layer.

6. A patent hierarchy classifying apparatus characterized by comprising: The method comprises the following steps: a patent text information receiving module is configured to receive patent text information to be classified, wherein the patent text information comprises a title and / or an abstract and / or claims of a patent, the claims comprising at least one independent claim and / or at least one independent claim and at least one dependent claim of the at least one independent claim; a preprocessing module is configured to preprocess the patent text information to obtain preprocessed patent text information; The classification module is configured to input the preprocessed patent text information into a patent classification model to obtain patent classification results at each level and corresponding prediction probability values. The patent classification model independently classifies the preprocessed patent text information at each level and obtains the patent classification results at each level and the corresponding prediction probability values based on the classification results at each level. The prediction probability value corresponding to a higher level classification result is greater than or equal to the prediction probability value corresponding to a lower level classification result. The patent classification model includes a plurality of independent classification sub-models corresponding to the levels of patent classification. The independent classification sub-models classify the preprocessed patent text information to obtain classification results at the levels of patent classification corresponding to the independent classification sub-models. The independent classification sub-models include an embedding layer, a key feature layer, a local deep network layer, a global feature layer, and a classification layer. The embedding layer obtains word vectors based on the preprocessed patent text information. The key feature layer extracts key feature vectors from the word vectors. The local deep network layer extracts local key feature vectors from the key feature vectors. The global feature layer extracts global key feature vectors from the key feature vectors. The classification layer fuses the local key feature vectors and the global key feature vectors to obtain fused feature vectors and classifies the fused feature vectors. The local deep network layer includes: three different parallel convolution layers; they are respectively a first convolution layer with a kernel size of 2 n, a second convolution layer with a kernel size of 3 n, a second convolution layer with a kernel size of 4 n, a third convolution layer with a kernel size of 4; wherein n is a dimension of the key feature vector; Correspondingly, the local deep network layer extracts local key feature vectors from the key feature vectors by: with a kernel size of 2 n, a first convolutional layer with 500 kernels of size 3 n, a second convolutional layer with 500 kernels of size 4 n, a third convolutional layer with 500 kernels, respectively convolving the key feature vectors to obtain bi-gram feature vectors, tri-gram feature vectors and four-gram feature vectors.

7. An electronic device, comprising: The electronic device includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the patent hierarchical classification method of any one of claims 1-5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the patent hierarchical classification method of any one of claims 1-5.

Images