Training sample labeling method and device, classification model training method and device

Abstract

The present application provides a training sample labeling method for training a classification model for overall classification of patents to be classified, the method comprising: based on the patent to be labeled, determining the patent feature vector and general search category information set corresponding to the patent to be labeled; based on the overall classification data, determining the overall category feature vector corresponding to each of the multiple overall categories included in the overall classification data; determining the comparison relationship between the multiple overall categories and the general search category; based on the patent feature vector, the general search category information set, the overall category feature vector corresponding to each of the multiple overall categories, and the comparison relationship, determining the overall category labeling information corresponding to the patent to be labeled. The training sample labeling method provided in the embodiment of the present application does not require manual labeling by professionals, and can quickly generate a large number of high-quality training samples for training classification models, which not only avoids strong reliance on professionalism, but also saves a lot of manpower, material resources and time costs.

Description

Technical Field

[0001] The present application relates to the field of patent classification technology, and specifically to a training sample annotation method and device, a classification model training method and device, an electronic device, and a computer-readable storage medium. Background Art

[0002] Classifying patents into strategic emerging industries, national economic industries, and high-tech industries is crucial to monitoring the development of various plans. Usually, a pre-trained classification model is used to classify patents into categories. However, training classification models requires a large number of training samples that have been labeled with comprehensive categories. The existing technology uses a large number of professionals to manually label, which is not only highly dependent on professionals, but also consumes a lot of manpower, material resources and time costs. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a training sample labeling method, a classification model training method, a training sample labeling device, a classification model training device, an electronic device and a computer-readable storage medium to solve the technical problem of low efficiency in the coordinated classification of patents in the prior art.

[0004] According to the first aspect of the embodiment of the present application, a training sample labeling method is provided, and the training samples are used to train a classification model for comprehensively classifying patents to be classified. The method includes: based on the patent to be marked, determining the patent feature vector and the general search category information set corresponding to the patent to be marked; based on the comprehensive classification data, determining the comprehensive category feature vectors corresponding to each of the multiple comprehensive categories included in the comprehensive classification data; determining the comparison relationship between the multiple comprehensive categories and the general search category; based on the patent feature vector, the general search category information set, the comprehensive category feature vectors corresponding to each of the multiple comprehensive categories, and the comparison relationship, determining the comprehensive category marking information corresponding to the patent to be marked.

[0005] In one embodiment, based on a patent feature vector, a general search category information set, and the coordinated category feature vectors corresponding to each of the multiple coordinated categories and a comparison relationship, the coordinated category marking information corresponding to the patent to be marked is determined, including: based on the patent feature vector and the coordinated category feature vectors corresponding to each of the multiple coordinated categories, determining the coordinated category information to be reviewed corresponding to the patent to be marked; based on the general search category information set and the comparison relationship, determining a candidate set of coordinated category information corresponding to the patent to be marked; if the candidate set of coordinated category information has an intersection with the coordinated category information to be reviewed, determining the intersection information between the candidate set of coordinated category information and the coordinated category information to be reviewed as the coordinated category marking information.

[0006] In one embodiment, based on the patent feature vector and the coordination category feature vectors corresponding to each of the multiple coordination categories, the coordination category information to be reviewed corresponding to the patent to be marked is determined, including: calculating the cosine distance between the patent feature vector and the coordination category feature vector corresponding to each of the multiple coordination categories to determine multiple cosine distance values ​​corresponding to the patent to be marked; if the largest cosine distance value among the multiple cosine distance values ​​corresponding to the patent to be marked is greater than a first threshold, then the coordination category corresponding to the largest cosine distance value is determined as the coordination category information to be reviewed corresponding to the patent to be marked.

[0007] In one embodiment, based on the patent to be marked, determining the patent feature vector corresponding to the patent to be marked includes: performing one-hot encoding operations on the title, abstract, claim specification, technical field, background technology, invention content and specific implementation method in the patent to be marked respectively, and determining the title one-hot vector, abstract one-hot vector, claim one-hot vector, specification one-hot vector, technical field one-hot vector, background technology one-hot vector, invention content one-hot vector and specific implementation method one-hot vector corresponding to the patent to be marked; based on the preset weight values ​​corresponding to the title one-hot vector, abstract one-hot vector, claim one-hot vector, specification one-hot vector, technical field one-hot vector, background technology one-hot vector, invention content one-hot vector and specific implementation method one-hot vector, performing weighted operation on the title one-hot vector, abstract one-hot vector, specification one-hot vector, technical field one-hot vector, background technology one-hot vector, invention content one-hot vector and specific implementation method one-hot vector, and determining the patent feature vector.

[0008] In one embodiment, based on the overall classification data, the overall classification feature vectors corresponding to each of the multiple overall classification data included in the overall classification data are determined, including: based on the overall classification data, the regional data corresponding to each of the multiple overall classification categories are determined; for each of the multiple overall classification categories, based on preset information, a one-hot encoding operation is performed on the regional data corresponding to the overall classification category, and the obtained one-hot vector is determined as the overall classification feature vector corresponding to the overall classification category.

[0009] In one embodiment, the coordinated classification data is strategic emerging industry classification data, or national economic industry classification data, or high-tech classification data, or application field classification data; multiple coordinated categories are multiple strategic emerging industry categories, or multiple national economic industry categories, or multiple high-tech categories, or multiple application field categories.

[0010] According to the second aspect of the embodiment of the present application, a classification model training method is provided, and the classification model is used to comprehensively classify patents to be classified, and the method includes: determining the patent to be marked and the coordinated category marking information corresponding to the patent to be marked, wherein the coordinated category marking information is determined based on the training sample labeling method of the first aspect mentioned above; training an initial network model based on the patent to be marked and the coordinated category marking information corresponding to the patent to be marked to obtain a classification model.

[0011] According to the third aspect of the embodiment of the present application, a patent classification method is provided for comprehensively classifying patents to be classified, and the classification method includes: inputting the patent to be classified into a classification model determined based on the classification model training method of the second aspect above, obtaining K predicted comprehensive categories corresponding to the patent to be classified and the probability values ​​corresponding to each of the K predicted comprehensive categories, wherein K is a positive integer; for each of the K predicted comprehensive categories, if the probability value corresponding to the predicted comprehensive category is greater than or equal to a preset probability threshold, then determining that the predicted comprehensive category is the comprehensive category to which the patent to be classified belongs, so as to determine the M comprehensive categories to which the patent to be classified belongs, where M is a positive integer less than or equal to K.

[0012] In one embodiment, the patent classification method also includes: determining the sequence information corresponding to the M coordinated categories to which the patent to be classified belongs based on the size of the probability values ​​corresponding to each of the M coordinated categories to which the patent to be classified belongs; and outputting the M coordinated categories to which the patent to be classified belongs according to the sequence information corresponding to each of the M coordinated categories to which the patent to be classified belongs.

[0013] According to a fourth aspect of an embodiment of the present application, a training sample labeling device is provided, wherein the training samples are used to train a classification model for comprehensively classifying patents to be classified, and the device includes: a first determination module, configured to determine, based on the patent to be marked, a patent feature vector and a general search category information set corresponding to the patent to be marked; a second determination module, configured to determine, based on the comprehensive classification data, the comprehensive category feature vectors corresponding to each of the multiple comprehensive categories included in the comprehensive classification data; a third determination module, configured to determine the comparison relationship between the multiple comprehensive categories and the general search category; and a fourth determination module, configured to determine the comprehensive category marking information corresponding to the patent to be marked based on the patent feature vector, the general search category information set, the comprehensive category feature vectors corresponding to each of the multiple comprehensive categories, and the comparison relationship.

[0014] According to the fifth aspect of the embodiment of the present application, a classification model training device is provided, characterized in that the classification model is used to comprehensively classify patents to be classified, and the device includes: a fifth determination module, configured to determine the patent to be marked and the coordinated category marking information corresponding to the patent to be marked, wherein the coordinated category marking information is determined based on the training sample labeling method of the first aspect mentioned above; a training module, configured to train an initial network model based on the patent to be marked and the coordinated category marking information corresponding to the patent to be marked to obtain a classification model.

[0015] According to the sixth aspect of the embodiment of the present application, a patent classification device is provided for classifying patents to be classified into comprehensive categories, and the classification device includes: a first acquisition module, configured to input the patent to be classified into a classification model determined based on the classification model training method of the second aspect, and obtain K predicted comprehensive categories corresponding to the patent to be classified and the probability values ​​corresponding to each of the K predicted comprehensive categories, wherein K is a positive integer; a sixth determination module, configured to determine the predicted comprehensive category as the comprehensive category to which the patent to be classified belongs if the probability value corresponding to the predicted comprehensive category is greater than or equal to a preset probability threshold for each of the K predicted comprehensive categories, so as to determine the M comprehensive categories to which the patent to be classified belongs, wherein M is a positive integer less than or equal to K.

[0016] According to the seventh aspect of the embodiments of the present application, there is provided an electronic device, comprising: a processor; and a memory, in which computer program instructions are stored, and when the computer program instructions are executed by the processor, the processor executes the methods of the first and second aspects described above.

[0017] According to an eighth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the methods of the first and second aspects described above.

[0018] The training sample labeling method provided in the embodiment of the present application is used to train a classification model for overall classification of patents to be classified, by determining the patent feature vector and general search category information set corresponding to the patent to be marked based on the patent to be marked; determining the overall category feature vector corresponding to each of the multiple overall categories included in the overall classification data based on the overall classification data; determining the comparison relationship between multiple overall categories and general search categories; determining the overall category labeling information corresponding to the patent to be marked based on the patent feature vector, the general search category information set, the overall category feature vector corresponding to each of the multiple overall categories, and the comparison relationship, so as to achieve the purpose of quickly generating a large number of high-quality training samples for training classification models without manual labeling by professionals. The use of the training sample labeling method provided in the embodiment of the present application not only avoids strong reliance on professionalism, but also saves a lot of manpower, material resources and time costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The figure is a flow chart of a training sample labeling method provided in one embodiment of the present application.

[0020] Figure 2 The figure is a flow chart of a training sample labeling method provided in one embodiment of the present application.

[0021] Figure 3 The figure is a flow chart of a training sample labeling method provided in one embodiment of the present application.

[0022] Figure 4 The figure is a flow chart of a training sample labeling method provided in one embodiment of the present application.

[0023] Figure 5 The figure is a flow chart of a training sample labeling method provided in one embodiment of the present application.

[0024] Figure 6 The figure is a flow chart of a training sample labeling method provided in one embodiment of the present application.

[0025] Figure 7 The figure is a flow chart of a classification model training method provided in one embodiment of the present application.

[0026] Figure 7a Shown is a block diagram of a Bert model provided in an embodiment of the present application.

[0027] Figure 8a Shown is a flow chart of a patent classification method provided in one embodiment of the present application.

[0028] Figure 8b Shown is a flow chart of a patent classification method provided in one embodiment of the present application.

[0029] Fig. 9 FIG. 1 is a schematic diagram of the structure of a training sample labeling device provided in one embodiment of the present application.

[0030] Fig.10 Shown is a schematic diagram of the structure of a fourth determination module provided in an embodiment of the present application.

[0031] Fig.11 Shown is a schematic diagram of the structure of a second determination module provided in an embodiment of the present application.

[0032] Fig.12 Shown is a schematic diagram of the structure of a first determination module provided in an embodiment of the present application.

[0033] Fig.13 Shown is a schematic diagram of the structure of a classification model training device provided in one embodiment of the present application.

[0034] Fig.14a Shown is a schematic diagram of the structure of a patent classification device provided in one embodiment of the present application.

[0035] Fig.14b Shown is a schematic diagram of the structure of a patent classification device provided in one embodiment of the present application.

[0036] Fig.15 Shown is a schematic structural diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0038] Exemplary training sample labeling method

[0039] Figure 1 The flowchart of the training sample labeling method provided in one embodiment of the present application is shown. The labeled training samples obtained by the training sample labeling method provided in the embodiment of the present application are used to train a classification model for comprehensively classifying the patents to be classified.

[0040] It should be noted that the overall classification of classified patents mentioned in the embodiments of the present application includes but is not limited to strategic emerging industry classification of classified patents, national economic industry classification of classified patents, high-tech classification of classified patents, application field classification of classified patents, etc.

[0041] like Figure 1As shown, the training sample labeling method includes the following steps.

[0042] S101: Based on the patent to be marked, determine the patent feature vector and general search category information set corresponding to the patent to be marked.

[0043] Specifically, the patent feature vector can represent the patent to be marked, and feature extraction of the patent to be marked can obtain the patent feature vector corresponding to the patent to be marked.

[0044] The general search category is the category used for general classification search. The general search categories include: International Patent Classification (IPC), European Patent Classification (ECLA), US Patent Classification (CCL), Japanese Classification (FI / F-term), Combined Patent Classification (CPC), etc. For example, the International Patent Classification (IPC Classification) is compiled according to the "Strasbourg Agreement on the International Patent Classification" signed in 1971, and is the only internationally used patent document classification. For the same patent to be marked, it may have multiple IPC classification numbers. The international patent category information set corresponding to the patent to be marked is the set of international patent classification numbers of the patent to be marked.

[0045] S102: Based on the overall classification data, determine the overall classification category feature vectors corresponding to each of the multiple overall classification categories included in the overall classification data.

[0046] Specifically, the overall classification data is data that can provide a basis for classification when the overall classification is performed. The overall classification data provides multiple overall categories, as well as the keywords or core contents covered by each of the multiple overall categories. For each of the multiple overall categories, feature extraction is performed on the keywords or core contents covered by the overall category to obtain an overall category characteristic vector that can characterize the overall category, thereby obtaining the overall category characteristic vectors corresponding to each of the multiple overall categories in the overall classification data.

[0047] More specifically, since the overall classification of classified patents includes strategic emerging industry classification of classified patents, national economic industry classification of classified patents, high-tech classification of classified patents, application field classification of classified patents, etc., the overall classification data includes strategic emerging industry classification data, national economic industry classification data or high-tech classification data, and application field classification data.

[0048] S103: Determine the mapping relationship between multiple coordinated categories and general search categories.

[0049] Specifically, the mapping relationship between multiple coordinated categories and general search categories is used to indicate which category a patent belonging to a certain coordinated category to be marked belongs to in the general search category. For example, if the coordinated category is the strategic emerging industry category and the general search category is the IPC category, the mapping relationship can indicate which IPC category a patent belonging to a certain strategic emerging industry category belongs to in the IPC category.

[0050] The overall classification and the general search category classify the patents to be marked from two different directions. There is a correspondence between these two classifications. Determining the correspondence between multiple overall classifications and general search categories provides a basis for the subsequent marking of the patents to be marked.

[0051] S104: Determine the coordinated category marking information corresponding to the patent to be marked based on the patent feature vector, the general search category information set, the coordinated category feature vectors corresponding to each of the multiple coordinated categories, and the comparison relationship.

[0052] Specifically, the coordinated category marking information corresponding to the patent to be marked is determined based on the patent feature vector corresponding to the patent to be marked, the general search category information set corresponding to the patent to be marked, the coordinated category feature vectors corresponding to each of the multiple coordinated categories in the coordinated classification, and the comparison relationship between the multiple coordinated categories and the general search category.

[0053] In the embodiment of the present application, by determining the patent feature vector and the general search category information set corresponding to the patent to be marked based on the patent to be marked; determining the overall category feature vector corresponding to each of the multiple overall categories included in the overall classification data based on the overall classification data; determining the comparison relationship between multiple overall categories and the general search category; determining the overall category marking information corresponding to the patent to be marked based on the patent feature vector, the general search category information set, the overall category feature vector corresponding to each of the multiple overall categories, and the comparison relationship, the purpose of quickly generating a large number of high-quality training samples for training classification models without manual marking by professionals is achieved. The training sample marking method provided in the embodiment of the present application not only avoids strong reliance on professionalism, but also saves a lot of manpower, material resources and time costs.

[0054] Figure 2 FIG. 1 is a flow chart of a training sample labeling method provided by an embodiment of the present application. Figure 2 As shown, based on the patent feature vector, the general search category information set, the coordinated category feature vectors corresponding to multiple coordinated categories and the comparison relationship, the steps of determining the coordinated category marking information corresponding to the patent to be marked include the following steps.

[0055] S201: Based on the patent feature vector and the coordinated category feature vectors corresponding to each of the multiple coordinated categories, determine the coordinated category information to be reviewed corresponding to the patent to be marked.

[0056] Specifically, the patent feature vector can represent the patent to be marked, and the coordinated category feature vectors corresponding to multiple coordinated categories can respectively represent multiple coordinated categories. The patent feature vector and multiple coordinated category feature vectors are matched to determine the coordinated category information to be reviewed, that is, the preliminary coordinated classification results of the patent to be marked are obtained by using feature vector matching, which provides a basis for subsequent more accurate classification results.

[0057] S202: Based on the general search category information set and the comparison relationship, determine the candidate set of coordinated category information corresponding to the patent to be marked.

[0058] Specifically, after determining the general search category information set corresponding to the patent to be marked, the reference relationship between multiple coordinated categories and general search categories is used to map out the coordinated categories corresponding to each general search category in the general search category information set in turn, and obtain the candidate set of coordinated category information corresponding to the patent to be marked. By using the reference relationship, another preliminary coordinated classification result of the patent to be marked is obtained, which provides a basis for the subsequent more accurate classification results.

[0059] S203: If the coordinated category information candidate set has an intersection with the coordinated category information to be reviewed, the intersection information between the coordinated category information candidate set and the coordinated category information to be reviewed is determined as the coordinated category marking information.

[0060] Specifically, considering the mapping relationship between multiple coordinated categories and general search categories in the coordinated classification, the mapping relationship is obtained by matching the keywords and core contents covered by each coordinated category with the keywords and core contents covered by each general search category. However, the keywords and core contents are relatively high-level and cannot completely cover the lower-level concepts, so that the accuracy of the coordinated category corresponding to the patent to be marked is low when initially matched according to the general search category information set corresponding to the patent to be marked. In addition, considering that although the patent feature vector can represent the patent to be marked, and multiple coordinated category feature vectors can represent the patent to be marked, some information will be lost during feature extraction, so that the preliminary coordinated classification results of the patent to be marked obtained by matching the feature vectors also need to be verified. The intersection of the two preliminary coordinated classification results can obtain more accurate coordinated category marking information.

[0061] In the embodiment of the present application, the preliminary overall classification result of the patent to be marked is obtained by using the feature vector matching method, and another preliminary overall classification result of the patent to be marked is obtained by using the comparison relationship between the overall classification and the general search category. The intersection of the two preliminary overall classification results is obtained to obtain more accurate overall classification labeling information. Combining the advantages of the preliminary overall classification results obtained by the two methods, more accurate overall classification labeling information is obtained.

[0062] Figure 3 FIG. 1 is a flow chart of a training sample labeling method provided by an embodiment of the present application. Figure 3 As shown, based on the patent feature vector and the coordinated category feature vectors corresponding to each of the multiple coordinated categories, the step of determining the coordinated category information to be reviewed corresponding to the patent to be marked includes the following steps.

[0063] S301: Calculate the cosine distance between the patent feature vector and the coordinated category feature vector corresponding to each coordinated category in multiple coordinated categories to determine multiple cosine distance values ​​corresponding to the patent to be marked.

[0064] Specifically, the cosine distance value between the patent feature vector and the coordinated category feature vector corresponding to each of the multiple coordinated categories is used to determine the matching degree between the patent to be marked and each of the multiple coordinated categories in the coordinated classification. The larger the cosine distance value, the greater the probability that the patent to be marked belongs to the coordinated category.

[0065] Step 302: If the largest cosine distance value among the multiple cosine distance values ​​corresponding to the patent to be marked is greater than the first threshold, the coordination category corresponding to the largest cosine distance value is determined as the coordination category information to be reviewed corresponding to the patent to be marked.

[0066] Specifically, considering that the larger the cosine distance value is, the greater the probability that the patent to be marked belongs to the coordination category corresponding to the cosine distance value is, that is, the coordination category corresponding to the largest cosine distance value is the coordination category to which the patent to be marked is most likely to belong, therefore, the largest cosine distance value among the multiple cosine distance values ​​corresponding to the patent to be marked is selected for further judgment. Specifically, it is judged whether the largest cosine distance value among the multiple cosine distance values ​​corresponding to the marked patent is greater than the first threshold value. If the largest cosine distance value among the multiple cosine distance values ​​corresponding to the patent to be marked is greater than the first threshold value, the coordination category corresponding to the largest cosine distance value is determined as the coordination category information to be reviewed corresponding to the patent to be marked.

[0067] It should be noted that if the maximum cosine distance value is too small, even if the coordination category corresponding to the maximum cosine distance value is the coordination category to which the patent to be marked is most likely to belong among all coordination categories, if the cosine distance value is too small, it indicates that the matching degree between the patent feature vector corresponding to the patent to be marked and the coordination category corresponding to the maximum cosine distance value is still too low, and the coordination category corresponding to the maximum cosine distance value cannot be determined as the coordination category information to be reviewed corresponding to the patent to be marked.

[0068] It should also be noted that the first threshold is determined according to the specific application scenario, and the embodiment of the present application does not limit the specific value.

[0069] In an embodiment of the present application, the cosine distance between the patent feature vector and the coordination category feature vector is calculated to determine the degree of match between the patent to be marked and each of the multiple coordination categories, and the coordination category with the highest matching degree and meeting the preset conditions is selected as the coordination category information to be reviewed corresponding to the patent to be marked.

[0070] Figure 4 FIG. 1 is a flow chart of a training sample labeling method provided by an embodiment of the present application. Figure 4 As shown, the step of determining the patent feature vector corresponding to the patent to be marked includes the following steps.

[0071] S401: Perform one-hot encoding operations on the title, abstract, claims, specification, technical field, background technology, invention content and specific implementation method of the patent to be marked respectively, and determine the title one-hot vector, abstract one-hot vector, claim one-hot vector, specification one-hot vector, technical field one-hot vector, background technology one-hot vector, invention content one-hot vector and specific implementation method one-hot vector corresponding to the patent to be marked.

[0072] Specifically, one-hot encoding is One hot encoding, and feature extraction can be performed through one-hot encoding operations. The title, abstract, claims, specification, technical field, background technology, content of the invention and specific implementation methods of the patent to be marked are subjected to one-hot encoding operations respectively to obtain the title one-hot vector, abstract one-hot vector, claims one-hot vector, specification one-hot vector, technical field one-hot vector, background technology one-hot vector, content of the invention one-hot vector and specific implementation method one-hot vector, which provide the basis for the subsequent acquisition of the patent feature vector corresponding to the patent to be marked.

[0073] S402: Based on the preset weight values ​​corresponding to the title one-hot vector, abstract one-hot vector, claim one-hot vector, specification one-hot vector, technical field one-hot vector, background technology one-hot vector, invention content one-hot vector and specific implementation method one-hot vector, weighted operations are performed on the title one-hot vector, abstract one-hot vector, specification one-hot vector, technical field one-hot vector, background technology one-hot vector, invention content one-hot vector and specific implementation method one-hot vector to determine the patent feature vector.

[0074] Specifically, considering that the length of the text content of the title, abstract, claims, instructions, technical field, background technology, invention content and specific implementation method in the entire patent to be marked is not the same, and the role they play in the overall classification is also different, therefore, it is necessary to consider the weight of each part when obtaining the patent feature vector of the patent to be marked. By presetting the weights of each part of the title, abstract, claims, instructions, technical field, background technology, invention content and specific implementation method, and based on the weights of each part, the title unique hot vector, abstract unique hot vector, claim unique hot vector, instruction unique hot vector, technical field unique hot vector, background technology unique hot vector, invention content unique hot vector and specific implementation method unique hot vector corresponding to the patent to be marked are weighted, and the role of each part in the overall classification is integrated to obtain the patent feature vector corresponding to the patent to be marked.

[0075] In the embodiment of the present application, a unique hot vector is established for each of the title, abstract, claim, specification, technical field, background technology, invention content and specific implementation method, and the weighted unique hot vector is obtained according to the preset weight as the patent feature vector. Compared with the unique hot encoding of the entire patent data in the prior art, the weight of each part is considered to highlight the part that should be focused on, thereby obtaining a more accurate patent feature vector, which provides a more accurate basis for subsequent matching with multiple coordinated category feature vectors.

[0076] It should be noted that the setting rule of the preset weight values ​​corresponding to the title unique hot vector, abstract unique hot vector, claim unique hot vector, specification unique hot vector, technical field unique hot vector, background technology unique hot vector, invention content unique hot vector and specific implementation method unique hot vector is: the more important and critical the title, abstract, claim, specification, technical field, background technology, invention content and specific implementation method are, the larger the corresponding weight value is. If any of these is considered not very important, its weight can be set to 0.

[0077] For example, in one embodiment, the title and abstract are considered more critical, and their weights are set greater than those of the claims and the description.

[0078] In another embodiment, if the background technology is considered unimportant, its corresponding weight may be set to 0. Figure 5 FIG. 1 is a flow chart of a training sample labeling method provided by an embodiment of the present application. Figure 5 As shown, based on the overall classification data, the step of determining the overall classification category feature vector corresponding to each of the multiple overall classification categories included in the overall classification data includes the following steps.

[0079] S501: Based on the overall classification data, determine the regional data corresponding to each of the multiple overall classification categories.

[0080] Specifically, for the overall classification data, the regional data corresponding to each of the multiple overall categories is the data corresponding to each of the multiple overall categories, which covers the keywords and core content of the overall category. According to the overall classification data, the regional data corresponding to each of the multiple overall categories is obtained respectively.

[0081] S502: For each of the multiple overall planning categories, based on preset information, perform a one-hot encoding operation on the regional data corresponding to the overall planning category, and determine the obtained one-hot vector as the overall planning category feature vector corresponding to the overall planning category.

[0082] Specifically, for the regional data of the overall planning category, a one-hot encoding operation is performed based on preset information, and the obtained one-hot vector is determined as the overall planning category feature vector corresponding to the overall planning category.

[0083] It should be noted that the preset information is information that can be used to modify the data corresponding to the overall planning category, wherein the modification of the data corresponding to the overall planning category includes deleting or adding keywords, increasing or decreasing the proportion of a certain keyword, and other modification behaviors.

[0084] In an embodiment of the present application, by performing one-hot encoding on the regional data corresponding to each of the multiple coordinated categories included in the coordinated classification data, preset information that can correct the regional data corresponding to the coordinated category is used during the encoding process to obtain a more accurate coordinated category feature vector, thereby providing a more accurate basis for subsequent matching with the patent feature vector.

[0085] Figure 6 The flowchart of the training sample annotation method provided by an embodiment of the present application is shown. The annotated training samples obtained by the training sample annotation method provided by this embodiment are used to train a classification model for strategic emerging industries classification of patents to be classified. The general search category is the IPC category (international patent category), the overall classification data is the strategic emerging industry classification data, and the overall category is the strategic emerging industry category.

[0086] like Figure 6 As shown, the specific steps of the training sample labeling method are as follows.

[0087] (1) Obtain the IPC classification number set corresponding to the patent to be marked. The "Reference Table of Strategic Emerging Industries Classification and International Patent Classification" provides a reference relationship between multiple strategic emerging industry categories and IPC classifications. According to the IPC classification number set corresponding to the patent to be marked and the reference relationship between the strategic emerging industry categories and IPC categories, obtain the candidate set of strategic emerging industry category information corresponding to the patent to be marked. The candidate set of strategic emerging industry category information is a preliminary strategic emerging industry classification result.

[0088] (2) Perform one-hot encoding operations on the title, abstract, claims, specification, technical field, background technology, invention content and specific implementation method in the patent to be marked respectively, determine the title one-hot vector, abstract one-hot vector, claim one-hot vector, specification one-hot vector, technical field one-hot vector, background technology one-hot vector, invention content one-hot vector and specific implementation method one-hot vector corresponding to the patent to be marked, preset the weights of each part of the title, abstract, claims, specification, technical field, background technology, invention content and specific implementation method, and based on the weights of each part, perform weighted operation on the title one-hot vector, abstract one-hot vector, claim one-hot vector, specification one-hot vector, technical field one-hot vector, background technology one-hot vector, invention content one-hot vector and specific implementation method one-hot vector corresponding to the patent to be marked, and obtain the patent feature vector corresponding to the patent to be marked.

[0089] The Classification of Strategic Emerging Industries provides data on multiple strategic emerging industry categories. The regional data corresponding to each strategic emerging industry category covers the keywords or core contents that patents that belong to the strategic emerging industry category should have. Keywords and weights are preset in advance. Based on the weights of keywords, the regional data corresponding to each strategic emerging industry category are uniquely encoded to obtain the unique hot vectors corresponding to multiple strategic emerging industry categories, and then obtain the characteristic vectors of multiple strategic emerging industry categories ( Figure 6 The characteristic vector of the strategic emerging industries category is 1~n, where n is a positive integer greater than or equal to 1).

[0090] (3) The cosine distance values ​​between the patent feature vector and the emerging industry category feature vectors 1 to n are calculated respectively, and it is determined whether the maximum cosine distance value is greater than the first threshold value. If the maximum cosine distance value is greater than the first threshold value, the strategic emerging industry category M corresponding to the maximum cosine distance value is determined as the strategic emerging industry category information to be reviewed, and another strategic emerging industry classification result of the patent to be marked is obtained.

[0091] (4) Determine whether the strategic emerging industry category M exists in the candidate set of strategic emerging industry category information. If so, determine the category information to be reviewed as the strategic emerging industry category marking information corresponding to the patent to be marked, and obtain a training sample. The training sample is used to train a classification model for strategic emerging industry classification of the patent to be classified.

[0092] Exemplary Classification Model Training Method

[0093] Figure 7 The figure is a flow chart of a classification model training method provided in an embodiment of the present application. The classification model trained by the classification model training method provided in an embodiment of the present application can comprehensively classify the patents to be classified.

[0094] like Figure 7 As shown, the classification model training method includes the following steps.

[0095] S701: Determine the patent to be marked and the overall category marking information corresponding to the patent to be marked.

[0096] Exemplarily, the overall category labeling information is determined based on the training sample labeling method provided in any of the above embodiments.

[0097] S702: Train an initial network model based on the patent to be marked and the overall category marking information corresponding to the patent to be marked to obtain a classification model.

[0098] Specifically, the initial network model is learned with the overall category labeling information obtained based on the above training sample labeling method as the goal. The overall category labeling information is the input reference value for training the initial network model, and the overall category prediction result of the initial network model is the output value. When there is a difference between the output value and the input reference value, it is necessary to adjust the parameters of the initial network model until the loss result is within the preset range, stop adjusting the parameters of the network model, and obtain the classification model.

[0099] More specifically, Figure 7a FIG. 1 is a block diagram of a Bert model provided by an embodiment of the present application. Figure 7aAs shown in the figure, when the initial classification model is a Bert (Bidirectional Encoder Representations from Transformer) model, the superposition of the token embedding module, the segmentation embedding module and the position embeddings Eembedding module in the Bert model is used to make the various parts of the patent (i.e., title, abstract, claims, and description parts) become independent parts, and the information of each part and the overall category tag is used to allow the initial classification model to learn how to determine the weight that each part should occupy in the final category result when learning to classify, so that the classification result obtained by the final classification model when performing overall category classification is a more accurate classification result that takes into account the weight of each part.

[0100] Exemplary Patent Classification Methodology

[0101] The patent classification method provided in the embodiment of the present application can comprehensively classify the patents to be classified.

[0102] Figure 8a FIG. 1 is a flow chart of a patent classification method provided by an embodiment of the present application. Figure 8a As shown, the classification model training method includes the following steps.

[0103] S801: Input the patent to be classified into the classification model determined by the classification model training method provided by any of the above embodiments to obtain K prediction coordination categories corresponding to the patent to be classified and the probability values ​​corresponding to each of the K prediction coordination categories.

[0104] Exemplarily, K is a positive integer.

[0105] Specifically, after the classification model is obtained by training using the classification model training method provided in any of the above embodiments, the patent to be classified is input into the classification model to obtain K prediction and coordination categories corresponding to the patent to be classified and the probability values ​​corresponding to each of the K prediction and coordination categories. For example, prediction and coordination category n1, prediction and coordination category n2, ..., prediction and coordination category nk, and the probability values ​​corresponding to each of the prediction and coordination categories n1, prediction and coordination category n2, ..., prediction and coordination category nk.

[0106] It should be noted that the probability value corresponding to a certain forecast coordination category among the K forecast coordination categories may be 0.

[0107] S802: For each of the K prediction coordination categories, if the probability value corresponding to the prediction coordination category is greater than or equal to the preset probability threshold, the prediction coordination category is determined to be the coordination category to which the patent to be classified belongs, so as to determine the M coordination categories to which the patent to be classified belongs.

[0108] Exemplarily, M is a positive integer less than or equal to K.

[0109] Specifically, through the preset probability threshold, for each of the K prediction coordination categories, the probability value corresponding to each prediction coordination category is compared with the preset probability threshold. If the probability value corresponding to a certain prediction coordination category is greater than or equal to the preset probability threshold, then the certain prediction coordination category is determined to be the coordination category to which the patent to be classified belongs. Similarly, from the K prediction coordination categories, all prediction coordination categories with probability values ​​greater than or equal to the preset probability threshold are determined as the coordination categories to which the patent to be classified belongs, that is, the M coordination categories to which the patent to be classified belongs are determined, and M is a positive integer less than or equal to K.

[0110] For example, if K is 10, after inputting the patent to be identified into the trained classification model, the predicted coordination category n1, predicted coordination category n2, ..., predicted coordination category n10 corresponding to the patent to be identified, and the probability values ​​corresponding to the predicted coordination category n1, predicted coordination category n2, ..., predicted coordination category n10 are obtained. Compare the probability value corresponding to each predicted coordination category with the preset probability threshold. If the probability values ​​corresponding to the predicted coordination category n1, predicted coordination category n2, and predicted coordination category n8 are all greater than the preset probability threshold, it is determined that the patent to be classified belongs to three coordination categories, namely, coordination category n1, coordination category n2, and coordination category n8.

[0111] It should be noted that the preset probability threshold is specifically set according to the specific application scenario.

[0112] In the embodiment of the present application, the patents to be classified are classified into categories through the above-mentioned patent classification method, so that a more comprehensive classification result corresponding to the patents to be classified can be obtained, providing users with a better user experience.

[0113] Figure 8b FIG. 1 is a flow chart of a patent classification method provided by an embodiment of the present application. Figure 8b As shown, the classification model training method also includes the following steps.

[0114] S803: Determine the order information corresponding to each of the M coordinated categories to which the patent to be classified belongs based on the magnitude of the probability values ​​corresponding to each of the M coordinated categories to which the patent to be classified belongs.

[0115] Specifically, based on the probability values ​​corresponding to the M coordinated categories to which the patent to be classified belongs, the M coordinated categories to which the patent to be classified belongs are sorted in descending order of probability values ​​to obtain the order information corresponding to the M coordinated categories to which the patent to be classified belongs. The larger the probability value corresponding to a coordinated category, the higher the corresponding order information.

[0116] S804: Output the M coordinated categories to which the patent to be classified belongs according to the sequence information corresponding to each of the M coordinated categories to which the patent to be classified belongs.

[0117] Specifically, according to the sequence information corresponding to the M coordinated categories to which the patent to be classified belongs, the coordinated category with the sequence information in front is output first, and then the coordinated category with the sequence information in the back is output, so that the M coordinated categories to which the patent to be classified belongs are displayed on the display terminal in order from large to small probability values.

[0118] To illustrate, following the above example, the relationship between the probability values ​​corresponding to the overall planning category n1, the overall planning category n2 and the overall planning category n8 is: the probability value corresponding to the overall planning category n1> the probability value corresponding to the overall planning category n8> the probability value corresponding to the overall planning category n2. The classification model is sent to the display terminal in the order of overall planning category n1, overall planning category n8 and overall planning category n2. The display terminal displays the classification results of the patent to be classified in the order of overall planning category n1, overall planning category n8 and overall planning category n2. The user can more intuitively understand which overall planning category among the M overall planning categories to which the patent to be classified belongs has a higher confidence level.

[0119] In an embodiment of the present application, the M coordinated categories to which the patent to be classified belongs are output through the above steps, so that the user can more intuitively understand which coordinated category among the M coordinated categories to which the patent to be classified belongs has a higher confidence level, thereby providing the user with a better user experience.

[0120] Exemplary training sample labeling device

[0121] Fig. 9 The diagram is a schematic diagram of the structure of a training sample labeling device provided in an embodiment of the present application. The training samples obtained by the training sample labeling device provided in an embodiment of the present application are used to train a classification model for comprehensively classifying patents to be classified. Fig. 9 As shown, the training sample labeling device 100 includes a first determination module 101 , a second determination module 102 , a third determination module 103 and a fourth determination module 104 .

[0122] The first determination module 101 is configured to determine, based on the patent to be marked, the patent feature vector and the general search category information set corresponding to the patent to be marked. The second determination module 102 is configured to determine, based on the overall classification data, the overall classification feature vectors corresponding to each of the multiple overall classifications included in the overall classification data. The third determination module 103 is configured to determine the comparison relationship between the multiple overall classifications and the general search category. The fourth determination module 104 is configured to determine the overall classification marking information corresponding to the patent to be marked based on the patent feature vector, the general search category information set, the overall classification feature vectors corresponding to each of the multiple overall classifications, and the comparison relationship.

[0123] In the embodiment of the present application, the first determination module 101 determines the patent feature vector and the general search category information set corresponding to the patent to be marked based on the patent to be marked, the second determination module 102 determines the overall category feature vector corresponding to each of the multiple overall categories included in the overall classification data based on the overall classification data; the third determination module 103 determines the comparison relationship between the multiple overall categories and the general search category; the fourth determination module 104 determines the overall category marking information corresponding to the patent to be marked based on the patent feature vector, the general search category information set, the overall category feature vector corresponding to each of the multiple overall categories, and the comparison relationship. The training sample labeling device provided in the embodiment of the present application can achieve the purpose of quickly generating a large number of high-quality training samples for training classification models, which not only avoids strong reliance on professionalism, but also saves a lot of manpower, material resources and time costs.

[0124] Fig.10 FIG. 4 is a schematic diagram of the structure of the fourth determination module provided in an embodiment of the present application. Fig.10 As shown, the fourth determining module 104 further includes a first determining unit 1041 , a second determining unit 1042 , and a first judging unit 1043 .

[0125] The first determination unit 1041 is configured to determine the coordinated category information to be reviewed corresponding to the patent to be marked based on the patent feature vector and the coordinated category feature vectors corresponding to each of the multiple coordinated categories; the second determination unit 1042 is configured to determine the candidate set of coordinated category information corresponding to the patent to be marked based on the general search category information set and the comparison relationship; the first judgment unit 1043 is configured to determine the intersection information between the candidate set of coordinated category information and the coordinated category information to be reviewed as the coordinated category marking information if there is an intersection between the candidate set of coordinated category information and the coordinated category information to be reviewed.

[0126] Fig.11 FIG. 2 is a schematic diagram showing the structure of a second determination module provided in an embodiment of the present application. Fig.11 As shown, the second determining unit 1042 further includes a calculating subunit 10421 and a first judging subunit 10422 .

[0127] The calculation subunit 10421 is configured to calculate the cosine distance between the patent feature vector and the coordination category feature vector corresponding to each coordination category in multiple coordination categories to determine multiple cosine distance values ​​corresponding to the patent to be marked; the first judgment subunit 10422 is configured to determine the coordination category corresponding to the largest cosine distance value as the coordination category information to be reviewed corresponding to the patent to be marked if the largest cosine distance value among the multiple cosine distance values ​​corresponding to the patent to be marked is greater than a first threshold.

[0128] Fig.12 FIG. 1 is a schematic diagram of the structure of a first determination module provided in an embodiment of the present application. Fig.12 As shown, the first determination module 101 further includes a first one-hot vector determination unit 1011 and a weighting unit 1012 .

[0129] The first one-hot vector determination unit 1011 is configured to perform one-hot encoding operations on the title, abstract, claims, specification, technical field, background technology, invention content and specific implementation method in the patent to be marked, respectively, to determine the title one-hot vector, abstract one-hot vector, claim one-hot vector, specification one-hot vector, technical field one-hot vector, background technology one-hot vector, invention content one-hot vector and specific implementation method one-hot vector corresponding to the patent to be marked; the weighting unit 1012 is configured to perform weighted operations on the title one-hot vector, abstract one-hot vector, claim one-hot vector, specification one-hot vector, technical field one-hot vector, background technology one-hot vector, invention content one-hot vector and specific implementation method one-hot vector based on the preset weight values ​​corresponding to the title one-hot vector, abstract one-hot vector, claim one-hot vector, specification one-hot vector, technical field one-hot vector, background technology one-hot vector, invention content one-hot vector and specific implementation method one-hot vector, to determine the patent feature vector.

[0130] Fig.12 FIG. 2 is a schematic diagram showing the structure of a second determination module provided in an embodiment of the present application. Fig.12 As shown, the second determination module 102 further includes a comprehensive category data determination unit 1021 and a second one-hot vector determination unit 1022 .

[0131] The overall classification data determination unit 1021 is configured to determine the regional data corresponding to each of the multiple overall classifications based on the overall classification data. The second one-hot vector determination unit 1022 is configured to perform a one-hot encoding operation on the regional data corresponding to the overall classification category for each of the multiple overall classifications based on preset information, and determine the obtained one-hot vector as the overall classification feature vector corresponding to the overall classification category.

[0132] The specific functions and operations of other modules in the above training sample labeling device are already in Figures 1 to 5The training sample labeling method described in detail is described in , so its repeated description will be omitted here.

[0133] Exemplary classification model training apparatus

[0134] Fig.13 The structure diagram of the classification model training device provided in one embodiment of the present application is shown. The classification model trained by the classification model training method provided in the embodiment of the present application can comprehensively classify the patents to be classified. Fig.13 As shown, the classification model training device 200 includes a fifth determination module 205 and a training module 206 .

[0135] The fifth determination module 205 is configured to determine the patent to be marked and the coordinated category marking information corresponding to the patent to be marked, wherein the coordinated category marking information is determined based on the training sample labeling method provided in any of the above embodiments. The training module 206 is configured to train the initial network model based on the patent to be marked and the coordinated category marking information corresponding to the patent to be marked to obtain a classification model.

[0136] The classification model trained by the classification model training device provided by the embodiment of the present application can accurately predict the overall category of the patent to be classified.

[0137] The specific functions and operations of other modules in the above classification model training device are already in Figure 7 The classification model training method described in detail is described in , so its repeated description will be omitted here.

[0138] Exemplary Patent Classification Device

[0139] The patent classification device provided in the embodiment of the present application is used to comprehensively classify the patents to be classified.

[0140] Fig.14a The figure is a schematic diagram of the structure of the patent classification device provided by one embodiment of the present application. Fig.14a As shown, the patent classification device 400 includes: a first obtaining module 401 and a sixth determining module 402.

[0141] The first acquisition module is configured to input the patent to be classified into a classification model determined by the classification model training method provided by any of the above embodiments, and obtain K prediction coordination categories corresponding to the patent to be classified and the probability values ​​corresponding to each of the K prediction coordination categories, wherein K is a positive integer; the sixth determination module is configured to determine that the prediction coordination category is the coordination category to which the patent to be classified belongs if the probability value corresponding to the prediction coordination category is greater than or equal to a preset probability threshold for each of the K prediction coordination categories, so as to determine the M coordination categories to which the patent to be classified belongs, wherein M is a positive integer less than or equal to K.

[0142] Fig.14b The figure is a schematic diagram of the structure of the patent classification device provided by one embodiment of the present application. Fig.14b As shown, the classification model device 400 also includes a seventh determination module 403 and an output module 404 .

[0143] The seventh determination module 402 is configured to determine the order information corresponding to the M coordinated categories to which the patent to be classified belongs based on the magnitude of the probability values ​​corresponding to the M coordinated categories to which the patent to be classified belongs. The output module 403 is configured to output the M coordinated categories to which the patent to be classified belongs according to the order information corresponding to the M coordinated categories to which the patent to be classified belongs.

[0144] Exemplary Electronic Devices

[0145] Fig.15 FIG. 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Fig.15 As shown, electronic device 300 includes one or more processors 310 and memory 320 .

[0146] The processor 310 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 300 to perform desired functions.

[0147] The memory 320 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 310 may execute the program instructions to implement the training sample labeling method of each embodiment of the present application described above or and / or other desired functions.

[0148] In one example, the electronic device 300 may further include: an input device 330 and an output device 340 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0149] Of course, to simplify, Fig.15 Only some of the components related to the present application in the electronic device 300 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 300 may also include any other appropriate components.

[0150] Exemplary computer program products and computer-readable storage media

[0151] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the training sample labeling method provided according to the various embodiments of the present application described in the above "Exemplary training sample labeling method" section of this specification, or the steps of the classification model training method provided according to the various embodiments of the present application described in the above "Exemplary classification model training method" section, or the steps of the patent classification method provided according to the various embodiments of the present application described in the above "Exemplary patent classification method" section.

[0152] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and also conventional step-by-step programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0153] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the processor executes the steps of the training sample labeling method provided according to the various embodiments of the present application described in the above “Exemplary training sample labeling method” section of this specification, or the steps of the classification model training method provided according to the various embodiments of the present application described in the “Exemplary classification model training method” section, or the steps of the patent classification method provided according to the various embodiments of the present application described in the “Exemplary patent classification method” section.

[0154] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0155] It should be noted that the above examples are only specific embodiments of the present application. Obviously, the present application is not limited to the above examples, and there are many similar variations. All variations directly derived or associated from the contents disclosed in the present application by those skilled in the art should fall within the protection scope of the present application.

[0156] It should be understood that the qualifiers such as first and second mentioned in the embodiments of the present application are only used to more clearly describe the technical solutions of the embodiments of the present application and cannot be used to limit the scope of protection of the present application.

[0157] The above are only preferred embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A training sample labeling method, characterized in that: The training samples are used to train a classification model for comprehensively classifying patents to be classified, and the method includes: Based on the patent to be marked, determining the patent feature vector and the general search category information set corresponding to the patent to be marked; Based on the overall classification data, determining the overall classification feature vectors corresponding to each of the plurality of overall classifications included in the overall classification data; Determine the mapping relationship between the plurality of coordinated categories and the general search categories; Determine the overall category marking information corresponding to the patent to be marked based on the patent feature vector, the general search category information set, the overall category feature vectors corresponding to each of the multiple overall categories, and the comparison relationship; The step of determining the patent feature vector corresponding to the patent to be marked based on the patent to be marked includes: Perform one-hot encoding operations on the title, abstract, claims, specification, technical field, background technology, invention content and specific implementation method of the patent to be marked respectively, and determine the title one-hot vector, abstract one-hot vector, claim one-hot vector, specification one-hot vector, technical field one-hot vector, background technology one-hot vector, invention content one-hot vector and specific implementation method one-hot vector corresponding to the patent to be marked; Based on the preset weight values ​​corresponding to the title one-hot vector, the abstract one-hot vector, the claim one-hot vector, the specification one-hot vector, the technical field one-hot vector, the background technology one-hot vector, the invention content one-hot vector and the specific implementation method one-hot vector, weighted operation is performed on the title one-hot vector, the abstract one-hot vector, the specification one-hot vector, the technical field one-hot vector, the background technology one-hot vector, the invention content one-hot vector and the specific implementation method one-hot vector to determine the patent feature vector; The step of determining, based on the overall classification data, the overall classification category feature vectors corresponding to each of the plurality of overall classification categories included in the overall classification data comprises: Based on the overall classification data, determine the regional data corresponding to each of the plurality of overall classification categories; For each of the multiple overall categories, a one-hot encoding operation is performed on the regional data corresponding to the overall category based on preset information, and the obtained one-hot vector is determined as the overall category feature vector corresponding to the overall category.

2. The training sample labeling method according to claim 1, characterized in that: The step of determining the coordinated category marking information corresponding to the patent to be marked based on the patent feature vector, the general search category information set, the coordinated category feature vectors corresponding to each of the plurality of coordinated categories, and the comparison relationship includes: Determine the to-be-reviewed coordination category information corresponding to the to-be-marked patent based on the patent feature vector and the coordination category feature vectors corresponding to each of the plurality of coordination categories; Based on the general search category information set and the comparison relationship, determine a candidate set of coordinated category information corresponding to the patent to be marked; If the coordinated category information candidate set has an intersection with the coordinated category information to be reviewed, the intersection information between the coordinated category information candidate set and the coordinated category information to be reviewed is determined as the coordinated category label information.

3. The training sample labeling method according to claim 2, characterized in that: The step of determining the to-be-reviewed coordinated category information corresponding to the to-be-marked patent based on the patent feature vector and the coordinated category feature vectors corresponding to each of the plurality of coordinated categories includes: Calculating the cosine distance between the patent feature vector and the overall category feature vector corresponding to each overall category in the multiple overall categories to determine multiple cosine distance values ​​corresponding to the patent to be marked; If the largest cosine distance value among the multiple cosine distance values ​​corresponding to the patent to be marked is greater than the first threshold, the coordination category corresponding to the largest cosine distance value is determined as the coordination category information to be reviewed corresponding to the patent to be marked.

4. The training sample labeling method according to any one of claims 1 to 3, characterized in that: The coordinated classification data is strategic emerging industry classification data, or national economic industry classification data, or high-tech classification data, or application field classification data; the multiple coordinated categories are multiple strategic emerging industry categories, or multiple national economic industry categories, or multiple high-tech categories, or multiple application field categories.

5. A classification model training method, characterized in that: The classification model is used to comprehensively classify the patents to be classified, and the method includes: Determine a patent to be marked and the overall category marking information corresponding to the patent to be marked, wherein the overall category marking information is determined based on the training sample marking method according to any one of claims 1 to 4; An initial network model is trained based on the patent to be marked and the overall category marking information corresponding to the patent to be marked to obtain the classification model.

6. A patent classification method, characterized in that: It is used to classify patents to be classified in a comprehensive manner, and the classification method includes: Input the patent to be classified into the classification model determined based on the classification model training method described in claim 5, and obtain K prediction and coordination categories corresponding to the patent to be classified and the probability values ​​corresponding to each of the K prediction and coordination categories, wherein K is a positive integer; For each of the K prediction coordination categories, if the probability value corresponding to the prediction coordination category is greater than or equal to a preset probability threshold, the prediction coordination category is determined to be the coordination category to which the patent to be classified belongs, so as to determine the M coordination categories to which the patent to be classified belongs, where M is a positive integer less than or equal to K.

7. The patent classification method according to claim 6, characterized in that: Also includes: Determine the order information corresponding to each of the M coordinated categories to which the patent to be classified belongs based on the magnitude of the probability values ​​corresponding to each of the M coordinated categories to which the patent to be classified belongs; According to the sequence information corresponding to the M coordinated categories to which the patent to be classified belongs, the M coordinated categories to which the patent to be classified belongs are output.

8. A training sample labeling device, characterized in that: The training samples are used to train a classification model for comprehensively classifying patents to be classified, and the device includes: A first determination module is configured to determine, based on the patent to be marked, a patent feature vector and a general search category information set corresponding to the patent to be marked; A second determination module is configured to determine, based on the overall classification data, the overall classification feature vectors corresponding to each of the plurality of overall classifications included in the overall classification data; A third determination module is configured to determine the comparison relationship between the plurality of overall planning categories and the general search categories; A fourth determination module is configured to determine the coordinated category marking information corresponding to the patent to be marked based on the patent feature vector, the general search category information set, the coordinated category feature vectors corresponding to each of the plurality of coordinated categories, and the comparison relationship; The step of determining the patent feature vector corresponding to the patent to be marked based on the patent to be marked includes: Perform one-hot encoding operations on the title, abstract, claims, specification, technical field, background technology, invention content and specific implementation method of the patent to be marked respectively, and determine the title one-hot vector, abstract one-hot vector, claim one-hot vector, specification one-hot vector, technical field one-hot vector, background technology one-hot vector, invention content one-hot vector and specific implementation method one-hot vector corresponding to the patent to be marked; Based on the preset weight values ​​corresponding to the title one-hot vector, the abstract one-hot vector, the claim one-hot vector, the specification one-hot vector, the technical field one-hot vector, the background technology one-hot vector, the invention content one-hot vector and the specific implementation method one-hot vector, weighted operation is performed on the title one-hot vector, the abstract one-hot vector, the specification one-hot vector, the technical field one-hot vector, the background technology one-hot vector, the invention content one-hot vector and the specific implementation method one-hot vector to determine the patent feature vector; The step of determining, based on the overall classification data, the overall classification category feature vectors corresponding to each of the plurality of overall classification categories included in the overall classification data comprises: Based on the overall classification data, determine the regional data corresponding to each of the plurality of overall classification categories; For each of the multiple overall categories, a one-hot encoding operation is performed on the regional data corresponding to the overall category based on preset information, and the obtained one-hot vector is determined as the overall category feature vector corresponding to the overall category.

9. A classification model training device, characterized in that: The classification model is used to comprehensively classify the patents to be classified, and the device includes: A fifth determination module is configured to determine a patent to be marked and overall category marking information corresponding to the patent to be marked, wherein the overall category marking information is determined based on the training sample labeling method according to any one of claims 1 to 4; The training module is configured to train an initial network model based on the patent to be marked and the overall category marking information corresponding to the patent to be marked to obtain the classification model.

10. A patented classification device, characterized in that, Used to comprehensively classify patents to be classified, the classification device includes: A first obtaining module is configured to obtain K prediction and coordination categories corresponding to the patent to be classified and probability values ​​corresponding to each of the K prediction and coordination categories based on the classification model determined by the classification model training method according to claim 5, wherein K is a positive integer; The sixth determination module is configured to determine, for each of the K prediction coordination categories, that the prediction coordination category is the coordination category to which the patent to be classified belongs if the probability value corresponding to the prediction coordination category is greater than or equal to a preset probability threshold, so as to determine the M coordination categories to which the patent to be classified belongs, where M is a positive integer less than or equal to K.

11. An electronic device, comprising: processor; as well as A memory, in which computer program instructions are stored, and when the computer program instructions are executed by the processor, the processor is caused to perform the method according to any one of claims 1 to 7. 12 . A computer-readable storage medium having computer program instructions stored thereon, wherein when the computer program instructions are executed by a processor, the processor is caused to perform the method according to claim 1 .

Images