Patent theme clustering method based on reinforcement learning fine-tuning semantic vector model

Abstract

The invention discloses a patent theme clustering method based on a reinforcement learning fine-tuning semantic vector model, and belongs to the technical field of patent theme clustering analysis, and the method comprises the steps: obtaining a patent text set in a related field, and extracting structured and unstructured text information; key parameters are initialized; carrying out vectorization processing on the structured patent text information; performing vectorization processing on the unstructured patent text information based on a BGE semantic vector model; carrying out feature fusion and calculating a feature mean vector of the whole patent text set; carrying out patent theme clustering on the patent text set based on a K-means clustering algorithm; calculating an evaluation index of a clustering result; key parameter derivatives are calculated based on the PG theorem in reinforcement learning, and key parameters are updated; and calculating and outputting a clustering result based on the latest key parameters. According to the method, the accuracy of non-English patent text information vectorization and the accuracy of the patent theme clustering result are improved, and the objectivity of the patent theme clustering result is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of patent subject clustering analysis, and in particular to a patent subject clustering method based on a reinforcement learning fine-tuning semantic vector model. Background Art

[0002] In recent years, with the huge increase in the number of invention patents, companies are in urgent need of subject classification and value assessment of large quantities of patents in related fields to realize high-value patent mining, and further guide companies to make layouts in the field of scientific and technological innovation and contribute stronger technological driving force.

[0003] The first step in high-value patent mining is to scientifically identify and classify topics from the huge amount of patents in the field. At present, on the one hand, manual experience is mainly used to form a technical spectrum or subject category based on expert experience. On the other hand, intelligent technology is mainly used to form subject classification mining results based on patent data through text subject recognition and clustering algorithms. For the manual experience method, the classification results are relatively stable, and there is often a strong logical relationship between the topics, but it is impossible to fully cover all patent contents. For the intelligent technology method, the classification results can fully cover the massive historical patent data, but based on different intelligent technology solutions, the quality of the classification results also varies greatly. The traditional text topic identification model is mainly the LDA (Latent Dirichlet Allocation) model, which obtains the feature word weight vector by performing statistical analysis on the word frequency of the document set, thereby realizing the topic classification of the text. For example, the Chinese patent application number 202210205214.7 discloses a product typicality feature mining method and system based on topic integrated clustering. It mainly vectorizes the unstructured text with high noise in a specific field by improving the LDA model. However, this method requires the use of word segmentation software tools to restore the part of speech and extract stems for non-English language texts. The word segmentation results affect the quality of text vectorization. In addition, the common clustering algorithm is mainly the K-means algorithm, whose core idea is to divide the data points into k Different clusters are formed, and each data point is assigned to the nearest cluster to achieve the purpose of clustering. However, the number of clusters in this algorithm is k It is a hyperparameter that depends on domain experience. The setting of this parameter will affect the quality of clustering results. Summary of the invention

[0004] Purpose of the invention: The technical problem to be solved by the present invention is to provide a patent topic clustering method based on a reinforcement learning fine-tuned semantic vector model in response to the deficiencies of the prior art.

[0005] In order to solve the above technical problems, the present invention discloses a patent topic clustering method based on a reinforcement learning fine-tuning semantic vector model, which specifically includes the following steps:

[0006] Step 1: Obtain a patent text set in the relevant field and extract structured text information and unstructured text information;

[0007] Step 2: Initialize key parameters;

[0008] Step 3: vectorize the structured text information;

[0009] Step 4: vectorize the unstructured patent text information based on the BGE semantic vector model;

[0010] Step 5: feature fusion and calculation of feature mean vector of the entire patent text set;

[0011] Step 6: Cluster the patent text set into patent topics based on the K-means clustering algorithm;

[0012] Step 7: Calculate the evaluation index of the clustering results;

[0013] Step 8: Calculate the derivative of key parameters based on the PG theorem in reinforcement learning, update the key parameters, and determine the loop exit condition. If it is not satisfied, return to step 3, otherwise exit the loop;

[0014] Step 9: Based on the latest key parameters, calculate and output the clustering results.

[0015] In step 1, the patent text set in the relevant field is a collection of all patent texts in a certain relevant field that have been screened and are to be classified by subject matter. Assume that the total number of patent texts in the collection is N ; In step 1, the structured text information includes the applicant information and inventor information in the patent text; in step 1, the unstructured text information includes the abstract content in the patent text.

[0016] In step 2, the key parameters include the embedding matrix ( and ), the neural network weight parameter vector in the BGE semantic vector model ( ), parameters generated in the K-means clustering algorithm The weight parameter vector of the policy neural network ( and ),in, , , and is obtained by random setting, It uses open source pre-trained general parameters.

[0017] In step 3, the specific steps of vectorizing the structured text information are as follows:

[0018] Step 3-1: Integrate all applicants and inventors in the patent text collection into a summary table, and calculate the total number of applicants. The total number of inventors is ;

[0019] Step 3-2: Encode the applicant vector and inventor vector for each patent in the patent text collection. Assume that The applicant vector after patent encoding is , the inventor vector is , , the encoding is as follows:

[0020]

[0021]

[0022] Among them, any element and The value is 0 or 1. Indicates the applicant summary table The applicant is The applicant of the patent is not the applicant of the patent. Indicates the inventor summary table The inventor is The inventor of a patent, otherwise he is not the inventor of the patent; j represents index.

[0023] Step 3-3: Calculate the applicant embedding vector and inventor embedding vector for each patent in the patent text collection. Assume that The applicant embedding vector of the patent is , the inventor embeds the vector , calculated as follows:

[0024]

[0025]

[0026] in, for OK The embedding matrix of the columns, for OK The embedding matrix of the columns, and The parameters in the two matrices are constantly updated and changed during the iterative calculation. They are obtained by random settings during initialization. Manually preset.

[0027] For step 3 above, the structured information such as applicants and inventors in the patent text contains important information that can improve the quality of patent subject clustering. For example, a specific applicant will focus on the technical direction of a specific subject. Similarly, inventors also have their own main research fields. Therefore, by vectorizing the applicant and inventor information in the patent text and using it as a component of the subsequent feature vector, it is helpful to learn the rules and improve the clustering quality.

[0028] In step 4, the specific method of vectorizing the unstructured patent text information based on the BGE semantic vector model is as follows: Assume that the first The unstructured information (abstract content) of the patent is recorded as ,Will Input into the BGE semantic vector model to obtain the unstructured information feature vector , the calculation process is expressed as follows:

[0029]

[0030] in, is a weight parameter vector The neural network model can output feature vectors by inputting the abstract content of the patent text. The open-source pre-trained parameters are used for initialization. The open-source parameter values ​​are obtained based on general text data training. The text vectorization results obtained based on the open-source weight parameters may not perform well in a specific professional field. Therefore, the parameter vector in the neural network model Changes will be continuously updated during iterative calculations.

[0031] For step 4 above, the traditional LDA model-based vectorization of non-English texts requires reliance on word segmentation software, such as the Jieba word segmentation tool, and the word segmentation result directly affects the semantic vectorization result. However, when the BGE semantic vector model is used, the semantic direct vectorization of non-English texts can be achieved, and the result performs best among the semantic vectorization models based on the same neural network. Furthermore, the present invention integrates the algorithm framework of reinforcement learning, learns patent samples in a specific field, and optimizes the general neural network parameters in the BGE semantic vector model. Adjustments are made to achieve more professional and accurate semantic vector conversion, which is an important innovation in this technical field.

[0032] In step 5, the features are integrated and the feature mean vector of the entire patent text set is calculated. The specific steps are as follows:

[0033] Step 5-1: vectorize the structured and unstructured text information of each patent in the patent text collection and perform vector concatenation. Assume that The vector after splicing the patent is , the splicing method is as follows:

[0034]

[0035] Step 5-2: is the mean vector, is the covariance ( is the preset parameter), constructs a normal distribution, and performs sampling to obtain the eigenvector , and calculate the sampling The probability density of ;

[0036] Step 5-3: Calculate the feature mean vector of the entire patent text set. The specific method is as follows: Assume yes The column vector of the row, The elements of a row are denoted by , then the feature mean vector of the entire patent text set is recorded as ,Too The column vector of the row, The elements of a row are denoted by , any row of elements The calculation method is as follows:

[0037]

[0038] in, is the total number of patent texts in the patent text set.

[0039] For step 5 above, concatenating and fusing the results of structured and unstructured text vectorization can enrich the feature information and help improve the clustering quality. In addition, according to the PG theorem in reinforcement learning, the feature vector and its probability density need to be obtained by constructing a normal distribution function and probability sampling to meet the requirements of parameter differentiation and differentiation, promote correct parameter feedback, and achieve effective convergence of the algorithm.

[0040] In step 6, the specific steps of clustering the patent text set into patent topics based on the K-means clustering algorithm are as follows:

[0041] Step 6-1: Set the parameters in the K-means clustering algorithm according to the strategy , the strategy is expressed as follows:

[0042]

[0043] in, is a weight parameter vector and The strategy neural network model is constructed by inputting the feature mean vector after all patents are fused. It can output a discrete probability distribution and then obtain parameters based on the discrete probability distribution. and its corresponding discrete probability , the parameter vector in the neural network model and It will be updated and changed continuously in iterative calculations, and obtained through random settings during initialization;

[0044] Step 6-2: Random Settings Center Points , where any center point Both The column vector of the row The elements of a row are denoted by ;

[0045] Step 6-3: Calculate the feature vector of each patent To each center point The Euclidean distance , the calculation method is as follows:

[0046]

[0047] Step 6-4: Assign each patent to the corresponding cluster. The specific assignment method is as follows: For any patent's feature vector , according to the calculated Euclidean distance, find it to The nearest one among the centers, i.e. , then the first The patent is assigned to In the cluster, until all patents are classified;

[0048] Step 6-5: Recalculate Center Points , for any center point Any row in Elements The calculation method is as follows:

[0049]

[0050] in, Indicates the latest cycle calculation classified to The patent text serial number set of the cluster, express The total number of patent texts in ;

[0051] Step 6-6: Repeat steps 6-3 to 6-5 until the preset maximum number of iterations is reached.

[0052] For the above step 6, by designing the weight parameter vector and The strategy neural network model is combined with the reinforcement learning algorithm framework to achieve parameter Unsupervised adaptive learning is conducive to enhancing the objectivity of clustering results.

[0053] Furthermore, the evaluation index of the clustering result in step S7 adopts the Dunn index , The larger the value, the better the clustering effect. The calculation method is as follows:

[0054]

[0055] in, Represents any two clusters and The degree of distance is calculated by and Take a sample point in each and , calculate the Euclidean distance , take the minimum Euclidean distance as the distance between the two clusters; Represents any cluster The coverage diameter is calculated by taking the cluster Any two sample points and , calculate the Euclidean distance , take the largest Euclidean distance as this cluster Covering diameter.

[0056] For step 7 above, the Dunn index can be used to comprehensively evaluate the quality of the clustering results. This index guides the direction of clustering optimization in the entire algorithm and is also an important indicator for judging the end of the algorithm cycle.

[0057] In step 8, the method of calculating the derivative of key parameters based on the PG theorem in reinforcement learning is as follows:

[0058]

[0059] in, Can be a key parameter , , , and Any one of Indicates the key parameters The derivative of Indicates the expected value of the content in the brackets. , It is the sample in step 5-2 The probability density of The parameters obtained in step 6-1 The discrete probability of .

[0060] In step 8, the specific method of updating the key parameters is as follows:

[0061]

[0062] in, Indicates the updated key parameter value, Indicates the key parameter value before updating. Can be a key parameter , , , and Any one of is the learning rate, Decrements in a fixed pattern across loop iterations.

[0063] In step 8, the loop exit condition is ,in, It represents the Dunn index value obtained by the latest round of iterative calculation. It represents the Dunn index value obtained by the previous round of iteration. is the preset threshold.

[0064] For step 8 above, set a reasonable learning rate Ability to balance the convergence speed and learning quality of reinforcement learning algorithms, especially setting parameters in a fixed-mode decreasing method , which enables the algorithm to accelerate convergence in the early stage of iteration and refine the learning rules in the later stage of iteration, so as to achieve the goal of maintaining a benign balance between convergence speed and learning quality.

[0065] Beneficial effects: The present invention uses the BGE semantic vector model to directly convert the patent content of non-English text into vectors. At the same time, when using the BGE semantic vector model, considering the professionalism of the text content in the relevant field, the key parameters in the BGE semantic vector model are fine-tuned using the reinforcement learning method to improve the accuracy of the vectorization of patent text information and the accuracy of the patent subject clustering results. In addition, reinforcement learning is further used to adjust the parameters in the traditional K-means clustering algorithm. Iterative adjustments solve the problem of relying on manual experience to set parameters The problem of patent theme clustering is solved, and the objectivity of the patent theme clustering results is enhanced. The technical problems that the existing technology cannot objectively confirm the number of themes and is difficult to directly process non-English language texts are solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0067] Figure 1 It is a schematic diagram of the overall method framework of the present invention;

[0068] Figure 2 is a numerical visualization diagram of the patent theme clustering results in the embodiment of the present invention;

[0069] Figure 3 It is a diagram showing the convergence effect of the overall algorithm on the Dunn index in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0071] The present invention proposes a patent topic clustering method based on a reinforcement learning fine-tuning semantic vector model, such as Figure 1 As shown, the complete specific steps will be described in detail below.

[0072] Step 1: In the patent search system, by searching the full-text keywords, screen the patent texts in the relevant field and obtain the patent text collection in the relevant field (assuming that the total number of patent texts in the collection is ), and extracts a collection of structured text information (patent applicants and inventors) and unstructured text information (patent abstracts).

[0073] Step 2: Initialize key parameters, including the embedding matrix ( and ), the neural network weight parameter vector in the BGE semantic vector model ( ), parameters generated in the K-means clustering algorithm The weight parameter vector of the policy neural network ( and ),in, , , and By random setting, Use open source pre-trained general parameters.

[0074] Step 3: Vectorize the structured text information:

[0075] Step 3-1: Integrate all applicants that appear in the patent text collection into an applicant summary table, and count the total number of applicants. , and then integrate all inventors that appear in the patent text collection into an inventor summary table, and count the total number of inventors ;

[0076] Step 3-2: Encode the applicant vector and inventor vector for each patent in the patent text collection. Assume that The applicant vector after patent encoding is , the inventor vector is , the encoding is as follows:

[0077]

[0078]

[0079] Among them, any element and The value is 0 or 1. Indicates the applicant summary table The applicant is The applicant of the patent is not the applicant of the patent. Indicates the inventor summary table The inventor is The inventor of the patent is not the inventor of the patent otherwise;

[0080] Step 3-3: Calculate the applicant embedding vector and inventor embedding vector for each patent in the patent text collection. Assume that The applicant embedding vector of the patent is , the inventor embeds the vector , calculated as follows:

[0081]

[0082]

[0083] in, for OK The embedding matrix of the columns, for OK The embedding matrix of the columns, and The parameters in the two matrices will be continuously updated and changed during the iterative calculation. It is manually preset and has little effect on the final result. It is usually 64.

[0084] Step 4: Vectorize unstructured data based on the BGE semantic vector model: Assume that The unstructured information (abstract content) of the patent is recorded as ,Will Input into the BGE semantic vector model to obtain the unstructured information feature vector , the calculation process is expressed as follows:

[0085]

[0086] in, is a weight parameter vector The neural network model can output feature vectors by inputting the abstract content of the patent text. The open-source pre-trained parameters are used for initialization. The open-source parameter values ​​are obtained based on general text data training. The text vectorization results obtained based on the open-source weight parameters may not perform well in a specific professional field. Therefore, the parameter vector in the neural network model Changes will be continuously updated during iterative calculations.

[0087] Step 5: Perform feature fusion for each patent in the patent text set and calculate the feature mean vector of the entire patent text set:

[0088] Step 5-1: vectorize the structured and unstructured text information and perform vector concatenation. Assume that The vector after splicing the patent is , the splicing method is as follows:

[0089]

[0090] Step 5-2: is the mean vector, is the covariance ( is the preset parameter), constructs a normal distribution, and performs sampling to obtain the eigenvector , and calculate the sampling The probability density of ;

[0091] Step 5-3: Calculate the feature mean vector of the entire patent text set. The specific method is as follows: Assume yes The column vector of the row, The elements of a row are denoted by , then the feature mean vector of the entire patent text set is recorded as ,Too The column vector of the row, The elements of a row are denoted by , any row of elements The calculation method is as follows:

[0092]

[0093] in, is the total number of patent texts in the patent text set.

[0094] Step 6: Cluster the patent text collection into patent topics based on the K-means clustering algorithm:

[0095] Step 6-1: Set the parameters in the K-means clustering algorithm according to the strategy , the strategy is expressed as follows:

[0096]

[0097] in, is a weight parameter vector and The strategy neural network model is constructed by inputting the feature mean vector after all patents are fused. It can output a discrete probability distribution and then obtain parameters based on the discrete probability distribution. and its corresponding discrete probability , the parameter vector in the neural network model and It will be updated and changed continuously in iterative calculations, and obtained through random settings during initialization;

[0098] Step 6-2: Random Settings Center Points , where any center point Both The column vector of the row The elements of a row are denoted by ;

[0099] Step 6-3: Calculate the feature vector of each patent To each center point The Euclidean distance , calculated as follows:

[0100]

[0101] Step 6-4: Assign each patent to the corresponding cluster. The specific assignment method is as follows: For any patent's feature vector , according to the calculated Euclidean distance, find it to The nearest one among the centers, i.e. , then the first The patent is assigned to In the cluster, until all patents are classified;

[0102] Step 6-5: Recalculate Center Points , for any center point Any row in Elements The calculation is as follows:

[0103]

[0104] in, Indicates the latest cycle calculation classified to The patent text serial number set of the cluster, express The total number of patent texts in ;

[0105] Step 6-6: Repeat steps 6-3 to 6-5 until the preset maximum number of iterations is reached.

[0106] Step 7: Calculate the evaluation index of the clustering results. The evaluation index uses the Dunn index , The larger the value, the better the clustering effect. The calculation method is as follows:

[0107]

[0108] in, Represents any two clusters and The degree of distance is calculated by and Take a sample point in each and , calculate the Euclidean distance , take the minimum Euclidean distance as the distance between the two clusters, Represents any cluster The coverage diameter is calculated by taking the cluster Any two sample points and , calculate the Euclidean distance , take the largest Euclidean distance as this cluster Covering diameter.

[0109] Step 8: Calculate the derivative of key parameters based on the PG theorem in reinforcement learning, update the key parameters, and determine the loop exit condition. If it is not satisfied, return to step 3, otherwise exit the loop.

[0110] Furthermore, the method of calculating the derivatives of key parameters based on the PG theorem in reinforcement learning is as follows:

[0111]

[0112] in, Can be a key parameter , , , and Any one of Indicates the key parameters The derivative of Indicates the expected value of the content in the brackets. , It is the sample in step 5-2 The probability density of The parameters obtained in step 6-1 The discrete probability of .

[0113] Furthermore, the key parameters are updated as follows:

[0114]

[0115] in, Indicates the updated key parameter value, Indicates the key parameter value before updating. Can be a key parameter , , , and Any one of is the learning rate, Decrements in a fixed pattern across loop iterations.

[0116] Furthermore, the loop exit condition is ,in, It represents the Dunn index value obtained by the latest round of iterative calculation. It represents the Dunn index value obtained by the previous round of iteration. is the preset threshold.

[0117] Step 9: Based on the latest key parameters, execute step 6 to calculate and output the clustering results.

[0118] Example:

[0119] Using the existing air traffic control patents in the database as a sample set, the total number of patents in the sample set is The total number of applicants is The total number of inventors is Due to the large amount of data and the complex calculation process, this example uses a patent in the sample set. The parameter calculation is explained as follows:

[0120] patent Vectorization of structured information: Applicant vector is a 59-dimensional 0-1 column vector, the inventor vector is a 2352-dimensional 0-1 column vector. Furthermore, in this embodiment, is a The matrix of is a The matrix is ​​embedded in the matrix and After the action, the applicant embeds the vector is a 64-dimensional non-0-1 column vector, the inventor embeds the vector is a 64-dimensional non-0-1 column vector;

[0121] patent Vectorization of unstructured information: After the abstract text of the patent is processed by the BGE semantic vector conversion model, is a 512-dimensional non-0-1 column vector;

[0122] Further, the above , and Concatenate and get vector is a 640-dimensional non-0-1 column vector;

[0123] Further, preset for The covariance matrix of , the rest of the elements are 0, and after sampling, the feature vector is obtained is a 640-dimensional non-0-1 column vector and its probability density , and then calculate the overall feature mean vector based on the feature vectors of all patents in the sample set , is a 640-dimensional non-0-1 column vector;

[0124] Furthermore, after multiple rounds of iterative calculations using the K-means algorithm and the overall reinforcement learning framework, the number of patent theme clusters is finally output. , , the numerical visualization of clustering results is as follows Figure 2 As shown, at the same time, this embodiment records the Dunn index after each round of iteration , its convergence effect is as follows Figure 3 shown.

[0125] After manual analysis, the thematic meanings represented by the 42 clustering results are shown in Table 1:

[0126] Table 1 Example results of patent theme clustering

[0127]

[0128] It can be seen from the above embodiments that the patent topic clustering method based on the reinforcement learning fine-tuning semantic vector model proposed in the embodiments of the present application is feasible.

[0129] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium can store a computer program, and when the computer program is executed by the data processing unit, the invention content of the patent subject clustering method based on the reinforcement learning fine-tuning semantic vector model provided by the present invention and some or all steps in each embodiment can be run. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0130] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of computer programs and their corresponding general hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention are essentially or the part that contributes to the prior art can be embodied in the form of a computer program, i.e., a software product, which can be stored in a storage medium and includes several instructions for enabling a device including a data processing unit (which can be a personal computer, a server, a single-chip microcomputer, a MUU or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0131] The present invention provides a patent subject clustering method based on a fine-tuned semantic vector model based on reinforcement learning. There are many methods and ways to implement the technical solution. The above is only a preferred implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention. All components not specified in this embodiment can be implemented using existing technologies.

Claims

1. A patent topic clustering method based on reinforcement learning fine-tuning semantic vector model, characterized in that: The steps include: Step 1: Obtain a patent text set in the relevant field and extract structured text information and unstructured text information; Step 2: Initialize key parameters; Step 3: vectorize the structured text information; Step 4: vectorize the unstructured patent text information based on the BGE semantic vector model; Step 5: feature fusion and calculation of feature mean vector of the entire patent text set; Step 6: Cluster the patent text set into patent topics based on the K-means clustering algorithm; Step 7: Calculate the evaluation index of the clustering results; Step 8: Calculate the derivative of key parameters based on the PG theorem in reinforcement learning, update the key parameters, and determine the loop exit condition. If it is not satisfied, return to step 3, otherwise exit the loop; Step 9: Based on the latest key parameters, calculate and output the clustering results.

2. The patent topic clustering method based on reinforcement learning fine-tuning semantic vector model according to claim 1, characterized in that: The structured text information described in step 1 includes the applicant information and inventor information in the patent text, and the unstructured text information includes the abstract content in the patent text.

3. The patent topic clustering method based on reinforcement learning fine-tuning semantic vector model according to claim 2 is characterized in that: The key parameters for initialization described in step 2 include initializing the embedding matrix and , neural network weight parameter vector in BGE semantic vector model , parameters generated in the K-means clustering algorithm The weight parameter vector of the policy neural network and ,in, , , and Obtained through random settings, Use open source pre-trained general parameters.

4. The patent topic clustering method based on reinforcement learning fine-tuning semantic vector model according to claim 3 is characterized in that: The vectorization processing of the structured text information described in step 3 includes the following steps: Step 3-1: integrate all applicants that appear in the patent text collection into an applicant summary table, and integrate all inventors that appear in the patent text collection into an inventor summary table; Step 3-2, perform applicant vector encoding and inventor vector encoding for each patent in the patent text collection to obtain an applicant vector and an inventor vector; Step 3-3: Calculate the applicant embedding vector based on the applicant vector and inventor vector and inventor embedding vector , where i represents the patent index in the patent text set.

5. The patent topic clustering method based on reinforcement learning fine-tuning semantic vector model according to claim 4 is characterized in that: The vectorization processing of the unstructured patent text information based on the BGE semantic vector model described in step 4 includes: Unstructured information of patents ,Will Input into the BGE semantic vector model to obtain the unstructured information feature vector , the calculation process is expressed as follows: in, is a weight parameter vector The neural network model can output a feature vector by inputting the abstract content of the patent text. The parameter vector in the neural network model Changes will be continuously updated during iterative calculations.

6. The patent topic clustering method based on reinforcement learning fine-tuning semantic vector model according to claim 5 is characterized in that: The feature fusion described in step 5 and calculation of the feature mean vector of the entire patent text set include the following steps: Step 5-1: vectorize the structured and unstructured text information of each patent in the patent text collection and perform vector concatenation. Suppose the first The vector after splicing the patent is , the splicing method is as follows: Step 5-2: is the mean vector, As covariance, construct a normal distribution and sample to obtain the eigenvector , and calculate the sampling The probability density of ,in For the preset parameters, yes The column vector of the row, The elements of a row are denoted by , ; Step 5-3: Calculate the feature mean vector of the entire patent text set ,Will No. The elements of a row are denoted by , Any row of elements The calculation method is as follows: in, is the total number of patent texts in the patent text set.

7. The patent topic clustering method based on reinforcement learning fine-tuning semantic vector model according to claim 6 is characterized in that: The patent subject clustering of the patent text set based on the K-means clustering algorithm described in step 6 includes the following steps: Step 6-1: Set the parameters in the K-means clustering algorithm according to the strategy ; Step 6-2: Random Settings Center Points ; Step 6-3: Calculate the feature vector of each patent To each center point The Euclidean distance ; Step 6-4: assign each patent to the corresponding cluster; Step 6-5: Recalculate Center Points ; Step 6-6: Repeat steps 6-3 to 6-5 until the preset maximum number of iterations is reached.

8. The patent topic clustering method based on reinforcement learning fine-tuning semantic vector model according to claim 7 is characterized in that: Step 6-1: Set the parameters of the K-means clustering algorithm according to the strategy include: in, is a weight parameter vector and The strategy neural network model is constructed by inputting the feature mean vector after all patents are fused. It can output a discrete probability distribution and then obtain parameters based on the discrete probability distribution. and its corresponding discrete probability , the parameter vector in the neural network model and Changes will be continuously updated during iterative calculations.

9. The patent topic clustering method based on reinforcement learning fine-tuning semantic vector model according to claim 8 is characterized in that: The evaluation index described in step 7 is the Dunn index , calculated as follows: in, Represents any two clusters and The degree of distance is calculated by and Take a sample point in each and , calculate the Euclidean distance , take the minimum Euclidean distance as the distance between the two clusters; Represents any cluster The coverage diameter is calculated by taking the cluster Any two sample points and , calculate the Euclidean distance , take the largest Euclidean distance as this cluster Covering diameter.

10. The patent topic clustering method based on reinforcement learning fine-tuning semantic vector model according to claim 9 is characterized in that: The calculation method of the key parameter derivatives based on the PG theorem in reinforcement learning described in step 8 is as follows: in, The key parameter , , , and Any one of Indicates the key parameters The derivative of Indicates the expected value of the content in the brackets. , It is sampling The probability density of Is to obtain parameters The discrete probability when Update the key parameters described in step 8 as follows: in, Indicates the updated key parameter value, Indicates the key parameter value before updating. The key parameter , , , and Any one of is the learning rate, Decrements in a fixed pattern across loop iterations.

Images