A patent intelligent recommendation method based on reinforcement learning

Abstract

The application discloses a kind of patent intelligent recommendation method based on reinforcement learning, comprising: according to the patent data collected, in combination with market dynamic data flow, determine technical field heat trend score and market capital attention score, in combination with user interactive behavior data flow, determine real-time popularity factor;Based on user demand, generate candidate patent set and basic semantic matching score, according to technical field heat trend score and market capital attention score, obtain real-time market factor, according to basic semantic matching score, market factor, real-time popularity factor and static quality score, obtain dynamic recommendation value score, generate recommendation list;In response to user real-time interactive behavior data flow, adjust dynamic recommendation value score, for dynamic value network incremental training, according to the training result, execute update deployment.The application improves the timeliness, accuracy and system adaptive ability of recommendation by introducing dynamic data flow and realizing multi-source information fusion.

Description

Technical Field

[0001] This invention belongs to the field of patent intelligent recommendation technology, specifically relating to a patent intelligent recommendation method based on reinforcement learning. Background Technology

[0003] Early patent recommendation systems, such as Chinese patent CN118861437A, were based on traditional natural language processing techniques (such as word vectors and named entity recognition) and machine learning algorithms (such as K-means clustering and SVM classification). These systems achieved matching by performing shallow semantic analysis and static classification on patent texts and enterprise needs. However, this method has inherent limitations: first, its semantic understanding is insufficient, making it difficult to handle complex technical terms, ambiguities, and long-distance dependencies in patent texts; second, once the matching model is trained, it becomes fixed and cannot perceive or adapt to dynamic changes in the external market environment.

[0004] To improve the accuracy of recommendations, subsequent technologies have attempted to introduce more multi-dimensional evaluation indicators. For example, Chinese patent CN119669447A proposed a personalized recommendation method for university patents. This method constructs a citation network and calculates static indicators such as citation span score, first score, and second score, combining these with text similarity for a comprehensive weighted ranking. While this method considers the technological influence and relevance of patents to some extent, its evaluation system has significant flaws. Its core scoring mechanism heavily relies on static attributes such as historical citation data of patents, essentially representing a summary of past patents and failing to reflect current and future market popularity and technological trends.

[0005] Therefore, multi-dimensional scoring methods based on static historical indicators are historically oriented rather than future-oriented. The patents they recommend may be technologically outdated and out of touch with current market demands, making it difficult to meet companies' needs for accurate identification of cutting-edge technologies and high-potential patents in a rapidly changing market environment. Summary of the Invention

[0006] This invention provides a patented intelligent recommendation method based on reinforcement learning. By integrating real-time market dynamic data streams and user interaction behavior data streams to construct a dynamic evaluation system, it solves the problems of recommendation lag caused by reliance on static historical data and the inability to dynamically optimize recommendation strategies according to market environment and user preferences in existing technologies.

[0007] The technical solution adopted in this invention is as follows: A patent intelligent recommendation method based on reinforcement learning includes: Based on the collected patent data and combined with market dynamic data streams, the heat trend score and market capital attention score of the technology field are determined, and the real-time popularity factor is determined by combining user interaction behavior data streams. Based on user needs, a candidate patent set and basic semantic matching score are generated. Real-time market factors are obtained based on the technical field popularity trend score and market capital attention score. Dynamic recommendation value score is obtained based on the basic semantic matching score, the market factors, the real-time popularity factor, and the static quality score, and a recommendation list is generated. In response to the real-time user interaction data stream, the dynamic recommendation value score is adjusted for incremental training of the dynamic value network, and an update deployment is performed based on the training results.

[0008] The method disclosed in this invention also has the following additional technical features: The market dynamics data stream is as follows: The market dynamics data stream includes news data streams and investment and financing data streams; Based on the patent data and the news data stream, the frequency of keywords in the historical time series corresponding to each technical field is obtained. Through the time series prediction model, the predicted frequency value of keywords corresponding to the technical field is obtained. Based on the correspondence between the patent data and the technical field, the technical field popularity trend score of the patent data is obtained. Based on the patent data and the investment and financing data flow, and according to the historical financing amount, a market capital attention score is obtained.

[0009] By combining user interaction behavior data streams, the real-time popularity factor is determined as follows: Based on the user interaction behavior data stream, user interaction behavior types are divided, and factor scores are assigned to each type to obtain a daily popularity score. Based on the daily popularity score, a real-time popularity factor is obtained by applying a decay weighting through an exponential decay model, wherein the half-life of the exponential decay model is determined according to the period of the popularity score.

[0010] Based on the technology sector trend score and market capital attention score, real-time market factors are obtained, specifically: Based on the trend of technological focus and the level of market capital attention, a weighted average is used to obtain real-time market factors. The weighting is determined based on the user's industry and adjusted according to the technical field corresponding to the patent data.

[0011] Based on the basic semantic matching score, the market factor, the real-time popularity factor, and the static quality score, the dynamic recommendation value score is obtained, specifically as follows: Based on the basic semantic matching score, the market factor, the real-time popularity factor, and the static quality score, a multi-layer dynamic value network is input, wherein the multi-layer dynamic value network includes a content quality channel and a market popularity channel. Through the content quality channel, a first abstract feature representation is obtained based on the basic semantic matching score and the static quality score; through the market popularity channel, a second abstract feature representation is obtained based on the market factor and the real-time popularity factor. Based on the first abstract feature representation and the second abstract feature representation, weights are generated through an attention subnetwork, fused through a fully connected layer, and a dynamic recommendation value score is obtained through an activation function.

[0012] The content quality channel and the market popularity channel are specifically as follows: Both the content quality channel and the market popularity channel include at least two neural network layers. The number of neural network layers and / or neurons in the market popularity channel is greater than that in the content quality channel.

[0013] Based on the first abstract feature representation and the second abstract feature representation, weights are generated through an attention sub-network, specifically as follows: Based on the first abstract feature representation and the second abstract feature representation, the initial weights are obtained through the attention subnetwork; Based on user needs, the system perceives contextual information and makes adjustments according to the user's industry and the technical field corresponding to the patent data.

[0014] After generating the recommendation list, it also includes: Generate random numbers based on the dynamic recommendation value score; If the random number is greater than the preset exploration rate, then the final recommendation list is determined. Otherwise, from the patent data outside the recommendation list, a candidate set is selected based on the dynamic recommendation value score, and a patent data is randomly selected from the candidate set to replace the patent data with the lowest dynamic recommendation value score in the recommendation list.

[0015] In response to the real-time user interaction data stream, the dynamic recommendation value score is adjusted as follows: Based on the real-time user interaction data stream, determine the real-time interaction type to adjust the dynamic recommendation value score; The adjusted dynamic recommendation value score will be used for incremental training of the dynamic value network.

[0016] The present invention also provides a computer-readable storage medium having a computer program stored thereon. The program implements the method when executed by the processor.

[0017] Due to the adoption of the above technical solution, the beneficial effects achieved by this invention are as follows: 1. In this invention, by combining market dynamic data streams and user interaction behavior data streams, changes in the external market environment and shifts in user attention can be captured in real time. This allows the generated technology sector trend score, market capital attention score, and real-time popularity factor to reflect the latest market trends and user interests. This overcomes the recommendation lag problem caused by relying on the historical static attributes of patents, enabling recommendation results to not only be based on the past value of patents but also to possess a forward-looking judgment on current hot topics and future potential.

[0018] The dynamic recommendation value score is calculated by integrating the basic semantic matching score (representing technology relevance), real-time market factors (representing the market environment), real-time popularity factors (representing group attention), and static quality scores (representing the patent's intrinsic value). This fusion mechanism avoids the one-sidedness of single-dimensional evaluation and enables a comprehensive evaluation of patents from multiple complementary dimensions such as technology relevance, market prospects, social attention, and patent quality, thereby generating a more comprehensive and accurate recommendation list.

[0019] Furthermore, by responding to real-time user interaction data streams, the dynamic recommendation value score is adjusted for incremental training of the dynamic value network. Based on the training results, updates are deployed, constructing a reinforcement learning closed loop. This allows the system to automatically adjust its internal evaluation model (dynamic value network) based on real user feedback, enabling the recommendation strategy to continuously optimize as user preferences and market trends change. This overcomes the technical limitation of models becoming fixed once deployed and unable to adapt, giving the system long-term learning capabilities and continuously improving user experience and recommendation performance.

[0020] In summary, by introducing dynamic data streams, achieving multi-source information fusion, and constructing a reinforcement learning loop, significant results were achieved in improving the timeliness, accuracy, and adaptive capabilities of recommendations. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart of the patent intelligent recommendation method based on reinforcement learning according to one embodiment of the present invention. Detailed Implementation

[0022] To more clearly illustrate the overall concept of the present invention, a detailed description will be provided below with reference to the accompanying drawings and examples.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0024] like Figure 1 As shown, a patent intelligent recommendation method based on reinforcement learning includes: S100: Based on the collected patent data and combined with market dynamic data flow, determine the technology field popularity trend score and market capital attention score, and combine with user interaction behavior data flow to determine the real-time popularity factor.

[0025] The core objective of this step is to build a real-time, dynamic patent external environment perception system to provide forward-looking market signals and user behavior insights for subsequent intelligent recommendation decisions. It aims to overcome the inherent limitations of relying solely on static internal patent attributes (such as citation counts and IPC classification numbers). By introducing and processing external dynamic data streams in real time, it can perceive changes in technological development trends, capital flows, and public attention, thereby ensuring that recommended patents are not only technologically relevant but also possess high market potential and timeliness.

[0026] It should be noted that the parallel acquisition of multi-source heterogeneous data adopts a distributed microservice architecture, deploying multiple dedicated data collectors, each connecting to different data sources, to ensure efficient and stable data acquisition.

[0027] Among them, patent data stream collection involves periodically pulling incremental patent data through the patent office's public API or commercial database interface, parsing and structuring key fields such as IPC classification number, claim items, and legal status.

[0028] Market dynamic data stream collection includes news streams, which utilizes a web crawler cluster to monitor and extract content from predefined authoritative industry news websites and government policy release platforms in real time. It also includes investment and financing data streams, which acquire investment and financing event information through commercial data service APIs (such as Qimingpian and ITjuzi), recording key information such as event amount, funding round, and relevant technology field.

[0029] User interaction behavior data stream collection involves deploying a tracking SDK on the system front end and collecting users' unconscious interaction logs in real time through message queues (such as Kafka), including behaviors such as search queries, patent clicks, dwell time on detail pages, and initiation of inquiries.

[0030] Real-time computation and quantification of dynamic features: Utilizing stream processing engines (such as Apache Flink) to process and extract features from the collected data stream in real time, transforming unstructured text information and user behavior into quantifiable dynamic metrics.

[0031] A higher technology sector trend score indicates a faster future increase in popularity for that technology sector. A higher market capital attention score indicates stronger capital investment and confidence in that sector. Both the technology sector trend score and the market capital attention score are obtained through dynamic market data streams. The real-time patent popularity factor represents the current level of user attention and is obtained through user interaction behavior data streams.

[0032] In the fast-paced market of technological innovation, the value of a patent is determined not only by its inherent technical content but also significantly influenced by the external market environment and public perception. Traditional static evaluation systems cannot capture these dynamic changes, leading to a disconnect between recommendations and market reality.

[0033] This step enables the assessment of market trends and user feedback, achieving environmental awareness in recommendations. By analyzing technology sector trend scores and market capital attention scores, it can recommend emerging technologies in their growth phase and potential sectors favored by capital, improving the timeliness and foresight of recommendations.

[0034] Furthermore, by leveraging the real-time patent popularity factor, technologies currently receiving widespread attention from peers can be automatically identified and recommended, enhancing the social acceptance and practicality of the recommendations. Moreover, the generated technology field trend score, market capital attention score, and real-time patent popularity factor provide reliable and information-rich input for subsequent dynamic recommendation value score calculations.

[0035] S200: Based on user needs, generate a candidate patent set and a basic semantic matching score. Based on the technical field popularity trend score and market capital attention score, obtain a real-time market factor. Based on the basic semantic matching score, the market factor, the real-time popularity factor, and the static quality score, obtain a dynamic recommendation value score and generate a recommendation list.

[0036] The core objective of this step is to construct an intelligent decision-making center capable of comprehensively evaluating patent value. It deeply integrates information from four dimensions: the patent's intrinsic technical relevance (basic semantic matching score), external market environment (real-time market factors), public attention (real-time popularity factors), and its own legal quality (static quality score). Through an advanced neural network model, it outputs a comprehensive, dynamic, and accurate dynamic recommendation value score. Its fundamental goal is to solve the problems of static, rigid, and one-dimensional scoring systems, achieving intelligent value discovery and generating personalized recommendation lists that truly meet users' current needs and market trends.

[0037] The request parsing and context construction employ parallel querying and context-aware techniques to quickly construct the complete input for this recommendation task. It receives user requests and parses out the user ID and the technical requirements described in natural language.

[0038] Semantic matching involves invoking a semantic matching service, inputting user requirements, and quickly retrieving a set of candidate patents from a massive patent database, returning a basic semantic matching score for each patent. Contextual acquisition involves querying information such as the user's industry attributes and company size from a user profile database.

[0039] Market factor calculation generates real-time market factors based on the technology sector's trend score and market capital attention score. It should be noted that the technology sector to which a technology belongs can be determined based on technological needs, and weights can be assigned to the technology sector's trend score and market capital attention score; there are no restrictions on this.

[0040] The forward propagation of dynamic value networks utilizes a multi-layered dynamic value network to replace the traditional linear weighting formula, enabling deep nonlinear feature fusion and value assessment.

[0041] The augmented calculation of the static quality score is performed by a lightweight rule engine, with the following input: The size of a patent family reflects the international distribution and protection level of patents. The number of claims is generally positively correlated with the scope of protection and stability of a patent; Legal status, such as valid authorization, is assigned a high score, while pending or invalid authorization is assigned a low score.

[0042] The system outputs a scalar value between 0 and 1 using a simple linear weighting or table lookup method. This design deliberately avoids complex citation network calculations in the comparison documents, focusing instead on readily available, objective legal quality indicators. While ensuring no core quality information is lost, it improves response efficiency and guarantees the simplicity and efficiency of the evaluation system.

[0043] For each patent in the candidate patent set, assemble its feature vector: Feature_Vector = [S_i, M_i, P_i, Q_i]. Where S_i is the basic semantic matching score, M_i is the real-time market factor, P_i is the real-time popularity factor, and Q_i is the static quality score.

[0044] Based on feature vectors, a dynamic recommendation value score is output through a dynamic value network.

[0045] A recommendation list is generated by filtering from the candidate patent set and sorting them simply according to their dynamic value scores. This initial recommendation list provides a foundation for possible subsequent interactions and optimizations. The candidate patent set is then sorted in descending order according to their dynamic recommendation value scores. The top N patents are selected to form the initial recommendation list.

[0046] The ultimate value of a patent is a combination of its technological, legal, market, and social (attention) value. Any assessment based on a single dimension is incomplete. This step learns the complex, non-linear interactions between different characteristics, achieving an organic integration of multi-dimensional value.

[0047] S300: Responds to the real-time user interaction data stream, adjusts the dynamic recommendation value score, uses it for incremental training of the dynamic value network, and performs update deployment based on the training results.

[0048] The core objective of this step is to construct a closed-loop intelligent system capable of self-learning and self-optimization. It captures real-time user feedback on recommendation results, transforming this feedback into training signals for model optimization. This feedback is then used to continuously and incrementally train and seamlessly update the core decision-making model—the dynamic value network. Its fundamental goal is to proactively adapt to shifts in user preferences and changes in market trends, thereby achieving continuous improvement and long-term stability in recommendation performance.

[0049] Feedback data collection and training sample construction employ a context-based refined feedback recording and reward allocation mechanism to transform raw user behavior into high-quality training samples usable for reinforcement learning.

[0050] Dynamic Value Network (DVN) incremental training employs an online learning or incremental learning paradigm, using newly generated feedback data to frequently update the parameters of the DVN model in small batches, enabling the model to learn the latest user preference patterns.

[0051] Hot-loading of models involves loading a copy of the currently running DVN model from the model repository as the base for incremental training. Mini-batch training uses a recently collected batch of training samples (such as data from the past 24 hours), calculating gradients using the mean squared error (MSE) or hinge loss as the loss function via backpropagation.

[0052] The core objective of optimization is to make the DVN model's predicted value score (V_i) for patents as close as possible to the true value (Reward) derived from user feedback. In other words, it aims to teach the model to predict user satisfaction. Through this process, the model parameters (including those of the attention subnetwork) are fine-tuned, thereby: Revise its judgment on the importance of different features (e.g., if it is found that users are paying more attention to market hotspots recently, automatically increase the implicit weight of M_i). Learn new user preference patterns that were not covered by the initial training data.

[0053] Model evaluation and hot update deployment: Establish an automated model deployment pipeline to ensure that new models can safely and seamlessly replace old models after performance meets the standards, achieving silent upgrades.

[0054] Offline evaluation involves quickly evaluating the newly trained model using a validation set containing historical data before deployment. The core metrics can be precision-recall or normalized discounted cumulative gain (NDCG).

[0055] A / B testing and small-scale deployment: If the new model performs no worse than the baseline model in offline evaluation, it enters the A / B testing phase. A small portion (e.g., 5%) of user traffic is redirected to the new model (Group B), while the remaining users continue to use the old model (Group A). ​​Key business metrics (such as click-through rate and consultation conversion rate) are compared in real time between Groups A and B.

[0056] A full hot update is triggered if Group B's core metrics are significantly better than Group A's. Lossless deployment techniques such as shadow mode or blue-green deployment are used to dynamically deploy the new model to all online servers without interrupting service, completing the iteration.

[0057] In a changing market and technological environment, user interests shift and market trends rotate. This step, through continuous learning from real user feedback, enables the system to closely follow highly timely market dynamics and changes in user preferences, effectively avoiding model drift issues where model performance degrades over time.

[0058] As a preferred embodiment of the present invention, the market dynamic data stream specifically comprises: The market dynamics data stream includes news data streams and investment and financing data streams; Based on the patent data and the news data stream, the frequency of keywords in the historical time series corresponding to each technical field is obtained. Through the time series prediction model, the predicted frequency value of keywords corresponding to the technical field is obtained. Based on the correspondence between the patent data and the technical field, the technical field popularity trend score of the patent data is obtained. Based on the patent data and the investment and financing data flow, and according to the historical financing amount, a market capital attention score is obtained.

[0059] This implementation aims to construct a refined and quantifiable market environment perception module. Its core objective is to generate a technology sector trend score (H_t) reflecting the prospects of technological development and a market capital attention score (C_t) reflecting capital attitudes by processing news text streams and investment and financing data streams in parallel. This design aims to transform fuzzy, unstructured market information into dynamic features with clear predictability and comparability that can be used for intelligent model calculations, thereby providing a solid and forward-looking external environment input for subsequent patent valuation.

[0060] The news data stream processing pipeline calculates the trend score (H_t) of technological field popularity. Through time series modeling and trend prediction, it transforms news sentiment into predicted values ​​of future technological popularity.

[0061] It should be noted that the acquired raw text is automatically cleaned (HTML tags and irrelevant symbols are removed), segmented, and invalid words are filtered based on the stop word list.

[0062] Technical field keyword extraction and mapping is performed on the pre-built "Technical Field-Keyword Mapping Dictionary" (which associates IPC categories with a series of core scientific and technological terms) to match keywords in the pre-processed text.

[0063] Time series construction and frequency calculation are performed, using days as the time window, to aggregate and calculate the daily weighted frequency of keywords for each technology field. The weighting rules may consider the authority of the news source; for example, news published by mainstream official media is given higher weight. Thus, a daily frequency time series {date: frequency} is constructed for each technology field.

[0064] Trend prediction and H_t score generation utilize the Prophet time series forecasting model, inputting historical frequency data for each technology sector over the past 180 days. The model automatically learns the periodicity and trend of the series and predicts the daily frequency value for the next 7 days. The average slope or first-order difference mean of the frequency change during the forecast period is calculated as the raw value of the trend strength. Finally, through Min-Max normalization, the trend strength values ​​of all technology sectors are mapped to the 0-1 interval to obtain the final technology sector popularity trend score (H_t). A higher H_t value indicates a faster increase in attention to that technology sector in the near future.

[0065] In addition, the investment and financing data flow processing pipeline calculates market capital attention score (C_t), transforming capital investment into a comparable attention indicator.

[0066] The process involves aggregating and scaling the funding amounts. By technology sector, the total funding amount for all investment and financing events over the past 30 days is summed to obtain the total funding amount for that sector. This total funding amount is then logarithmically scaled using the formula: log(1 + total amount). This step aims to eliminate significant differences in funding scale across different sectors (power-law distribution), making the data more consistent with a normal distribution and facilitating subsequent modeling.

[0067] The C_t score is generated by scaling the values ​​and then standardizing them using the Z-Score across all technology sectors, converting them to a standard normal distribution. For ease of understanding and application, the Z-Score value can be further mapped to the 0-1 interval using the Sigmoid function to obtain the final market capital attention score (C_t). The higher the C_t value, the stronger the capital market's confidence and investment in that technology sector.

[0068] In the field of technological innovation, popularity and capital investment are two of the most critical external signals for measuring the viability and commercial prospects of a technological direction. However, these signals are embedded in unstructured text and discrete financial events. This implementation method uses a systematic and automated data processing workflow to extract these raw signals into stable and reliable numerical features, which is the foundation for embedding market perception capabilities into recommendation systems. H_t (reflecting media and industry attention) and C_t (reflecting capital market confidence) depict the market environment from different perspectives, providing a complementary dual perspective. Combining the two can more comprehensively assess the market potential of a patented technology.

[0069] In a preferred embodiment of the present invention, the real-time popularity factor is determined by combining user interaction behavior data stream, specifically as follows: Based on the user interaction behavior data stream, user interaction behavior types are divided, and factor scores are assigned to each type to obtain a daily popularity score. Based on the daily popularity score, a real-time popularity factor is obtained by applying a decay weighting through an exponential decay model, wherein the half-life of the exponential decay model is determined according to the period of the popularity score.

[0070] This preferred embodiment aims to construct a dynamic evaluation index that can reflect the degree of user attention to a patent in real time. Its core objective is to analyze user interaction data streams, transforming discrete user clicks, dwell times, and favorites into a quantitative score that comprehensively reflects the value of these behaviors and decays reasonably over time. This design aims to provide a dynamically changing signal reflecting the current actual popularity of a patent, thereby compensating for the shortcomings of relying solely on content semantics and static quality assessments, and making recommendation results more closely aligned with users' actual interests and market focus.

[0071] Behavior type classification and weight setting: Consultation behavior (weight W_b=1.0), the highest weight, represents that the user has a clear intention to convert, which is a very strong positive signal; The act of adding items to favorites (weight W_b=0.8) has a high weight, indicating that users highly approve of the product and hope for further follow-up. Valid clicks (weight W_b=0.5), medium weight, are defined as clicks that occur when the user stays on the details page for more than a preset threshold (e.g., 30 seconds), indicating that the user has engaged in a substantial browsing activity; A regular click (weight W_b=0.2) is a basic weight representing initial interest. Negative feedback behavior (weight W_b=-0.5), such as when a user explicitly clicks "not interested," can be used as a penalty signal.

[0072] Daily popularity score is calculated on a daily basis. For each patent i, all interactions received on that day are counted. The daily popularity score (Daily_Score_i) is calculated using the following formula: Daily_Score_i = Σ(Number of occurrences for each behavior type × corresponding W_b). This aggregates dissimilar user behaviors into a single comparable daily scalar value.

[0073] The real-time score calculation based on the exponential decay model introduces an exponential decay function to simulate the natural decline process of user attention, so that the popularity score can reflect recent popularity without completely ignoring valuable historical signals.

[0074] The current decay weight of historical daily scores is calculated using the formula D(t) = e^(-λ*t). Where: t, the difference in the number of days between the current time and the date the action occurred; λ, the decay rate coefficient, is a key parameter of the model.

[0075] The half-life is determined based on the period of the popularity score, which can be specifically implemented as follows: For fields with rapid technological iteration (such as artificial intelligence and semiconductors), a shorter half-life (such as 2 days) is set to ensure that the popularity is updated quickly and keeps up with the hot topics; For fields with stable technological development (such as traditional machinery and basic materials), setting a longer half-life (such as 7 days) allows historical attention to have a more lasting impact.

[0076] By introducing an exponential decay model with a configurable half-life, the system is made highly sensitive to emerging hotspots (with significant impact from recent behavior) while avoiding the complete discarding of valuable historical information, thus enabling the P_i factor to evolve smoothly and reasonably.

[0077] λ is calculated based on the half-life T_half: λ = ln(2) / T_half. For example, if the half-life is 2 days, then λ≈0.347.

[0078] The real-time popularity factor calculation formula for patent i at the current moment is as follows: P_i_raw=Σ[Daily_Score_i(t)*e^(-λ*(t_current-t))]. This sums the values ​​of t over all dates t containing records of the patent activity within the past N days.

[0079] It should be noted that, in order to achieve global normalization of the output, cross-patent normalization is used to eliminate the scale differences in absolute values, thereby ensuring the fairness and availability of the popularity factor within the system.

[0080] Periodically (e.g., every 15 minutes), perform quantile normalization or max-min normalization on the P_i_raw scores of all patents, mapping them to a fixed interval of 0-1. Output the final, normalized real-time popularity factor (P_i) and write it to the feature library for later use.

[0081] User behavior provides the most direct and authentic feedback for validating recommendation effectiveness and discovering emerging trends. However, raw behavioral data is sparse, noisy, and changes over time. This implementation method uses a systematic quantification, aggregation, and decay mechanism to refine the raw behavioral data into a stable, comparable, and information-rich dynamic feature. This is a key step in effectively integrating user feedback into machine learning models and achieving data-driven optimization.

[0082] As a preferred embodiment of the present invention, real-time market factors are obtained based on the technical field trend score and the market capital attention score, specifically as follows: Based on the trend of technological focus and the level of market capital attention, a weighted average is used to obtain real-time market factors. The weighting is determined based on the user's industry and adjusted according to the technical field corresponding to the patent data.

[0083] This preferred embodiment aims to construct a market environment assessment model that can adaptively adjust based on user background and technical field characteristics. Its core objective is to integrate the technical field popularity trend score (H_t) and market capital attention score (C_t) into a more personalized, real-time market factor (M_i) that better fits specific application scenarios through a context-aware dynamic weight allocation mechanism. This allows the same patent to demonstrate different facets of its market value when facing users from different industry backgrounds.

[0084] Construct a weighted knowledge base that includes two dimensions: user industry attributes and technical field characteristics. Specifically, User industry dimension weight preset: Academic research institutions, configured with [α=0.7, β=0.3], show a significant bias towards technological trends (H_t) because they are more focused on cutting-edge technologies and future developments; Venture capital firms, with an allocation of [α=0.3, β=0.7], show a significant bias towards capital attention (C_t), as their decisions rely more on validation from the capital market and expected returns. For manufacturing companies, a balanced weighting of [α=0.5, β=0.5] is used because they need to balance technological advancement and commercial maturity. The strategic R&D department, with a budget of [α=0.6, β=0.4], should pay attention to technological trends while also appropriately considering capital flows.

[0085] In addition, the weighting of the technology field dimension has been adjusted: In emerging frontier fields (such as quantum computing), an additional +0.1 correction value is added to the basic weight, because at this time, technology trend signals are more forward-looking than capital signals; For mature application areas (such as traditional machinery), the basic weight is β plus a correction value of +0.1, so that capital attention at this time can better reflect the actual market demand. Capital-intensive sectors (such as biopharmaceuticals) are weighted at β plus a correction of 0.15, highlighting the critical role of capital investment.

[0086] During recommendation request processing, the final weight value is dynamically calculated based on the specific context. Contextual recognition is used to extract user industry information from user profiles. Semantic analysis is used to identify the technical fields users are interested in (mapped to IPC classifications) from their technical requirement descriptions.

[0087] Based on the user's industry, the base weights [α_base, β_base] are obtained from preset rules. Adjusted weight values ​​[Δα, Δβ] are obtained based on the characteristics of the technical field. The final weights are calculated: α = α_base + Δα, β = β_base + Δβ. The weights are then normalized to ensure α + β = 1.

[0088] Real-time market factor calculation to obtain the H_t and C_t scores corresponding to this technology field. Calculation of real-time market factor: M_i = α*H_t + β*C_t.

[0089] In real-world business environments, decision-makers from diverse backgrounds exhibit significant differences in their sensitivity to market signals. Applying fixed weighting to all users and all technology sectors fails to accurately reflect these differentiated assessment needs. This implementation introduces a dual-dimensional dynamic weighting mechanism, enabling market assessments to be tailored to specific application scenarios.

[0090] This implementation method can automatically identify user background and technical field characteristics, dynamically adjust the focus of market evaluation, and improve the scenario adaptability of the recommendation system. Moreover, the same technology presents differentiated market value to different users, making the recommendation results more in line with the user's actual decision-making logic, thus achieving personalized market evaluation.

[0091] In a preferred embodiment of the present invention, a dynamic recommendation value score is obtained based on the basic semantic matching score, the market factor, the real-time popularity factor, and the static quality score, specifically as follows: Based on the basic semantic matching score, the market factor, the real-time popularity factor, and the static quality score, a multi-layer dynamic value network is input, wherein the multi-layer dynamic value network includes a content quality channel and a market popularity channel. Through the content quality channel, a first abstract feature representation is obtained based on the basic semantic matching score and the static quality score; through the market popularity channel, a second abstract feature representation is obtained based on the market factor and the real-time popularity factor. Based on the first abstract feature representation and the second abstract feature representation, weights are generated through an attention subnetwork, fused through a fully connected layer, and a dynamic recommendation value score is obtained through an activation function.

[0092] This preferred embodiment aims to construct a core evaluation model capable of deeply understanding the multi-dimensional value of patents and achieving intelligent fusion decision-making. Its core objective is to replace the traditional linear weighted scoring model with a multi-layered dynamic value network (DVN) featuring differentiated processing channels and an attention fusion mechanism, thereby addressing the problems of singular feature processing methods and rigid fusion strategies in existing technologies. This design can automatically learn the complex nonlinear relationships between different features and dynamically adjust the importance of each feature dimension according to the specific context, ultimately outputting a dynamic recommendation value score (V_i) that accurately reflects the comprehensive value of the patent.

[0093] The differentiated channel design of the multi-layer dynamic value network (DVN) provides independent processing channels for feature groups with different physical meanings and distribution characteristics.

[0094] Specifically, the content quality channel and the market popularity channel are: Both the content quality channel and the market popularity channel include at least two neural network layers. The number of neural network layers and / or neurons in the market popularity channel is greater than that in the content quality channel.

[0095] Specifically, the content quality channel (processing intrinsic value) takes as input the basic semantic matching score (S_i) and the static quality score (Q_i), and the network structure consists of two fully connected layers. The first layer maps the 2D input to 8D using the ReLU activation function. The second layer maps the 8D features to 16D using the ReLU activation function.

[0096] This outputs the first abstract feature representation F_A (a 16-dimensional vector), capturing the deep nonlinear relationship between technological relevance and patent legal quality (e.g., patents with high S_i but low Q_i should be moderately suppressed).

[0097] In addition, the market popularity channel (processing external environmental value) takes into account real-time market factors (M_i) and real-time popularity factors (P_i). Its network structure consists of two fully connected layers, with more neurons than the content quality channel. The first layer maps the 2-dimensional input to 16 dimensions (twice the number of dimensions in the content quality channel) using the ReLU activation function. The second layer maps the 16-dimensional features to 32 dimensions using the ReLU activation function.

[0098] The output is a second abstract feature representation F_B (a 32-dimensional vector). This channel's stronger representational power is used to learn more complex and noisier dynamic patterns in market signals (e.g., high M_i but low P_i may indicate blue ocean opportunities).

[0099] An attention mechanism is introduced to achieve dynamic and adaptive allocation of importance among channels. Specifically, initial weights are obtained based on the first abstract feature representation and the second abstract feature representation, combined with an attention sub-network. Based on user needs, the system perceives contextual information and makes adjustments according to the user's industry and the technical field corresponding to the patent data.

[0100] F_A and F_B are concatenated into a fused feature vector F_combined (48 dimensions).

[0101] Design a lightweight attention subnetwork with input F_combined. The subnetwork structure is: a fully connected layer (48-dimensional input -> 8-dimensional output) + ReLU → a fully connected layer (8-dimensional -> 2-dimensional output) + Softmax. The Softmax layer ensures that the attention weights [α, β] of the output satisfy α + β = 1. α represents the importance of the content quality channel in the current context. β represents the importance of the market popularity channel in the current context. Calculate the weighted feature vector F_weighted = α * F_A + β * F_B. This step dynamically determines whether to rely more on the intrinsic quality or external popularity of the current patent feature based on its own characteristics.

[0102] The final value abstraction and calculation are performed using a deep neural network. The deep abstraction layer passes F_weighted through two fully connected layers (e.g., 64-dimensional -> 32-dimensional -> 16-dimensional) using the ReLU activation function to progressively extract high-level abstract features for value judgment. The output layer finally uses a single neuron with a Sigmoid activation function to restrict the output range to (0, 1), which serves as the dynamic recommendation value score (V_i) of this patent.

[0103] This implementation uses differentiated channels to process heterogeneous features and introduces an attention mechanism to achieve dynamic fusion, simulating and realizing complex value judgments. Through independent channels, the model can learn deep patterns of technical content / quality and market / popularity features in the most appropriate way, avoiding information confusion and model learning difficulties caused by simply concatenating features at the input layer, thus achieving deep and accurate processing of heterogeneous features.

[0104] In a preferred embodiment of the present invention, after generating the recommendation list, the method further includes: Generate random numbers based on the dynamic recommendation value score; If the random number is greater than the preset exploration rate, then the final recommendation list is determined. Otherwise, from the patent data outside the recommendation list, a candidate set is selected based on the dynamic recommendation value score, and a patent data is randomly selected from the candidate set to replace the patent data with the lowest dynamic recommendation value score in the recommendation list.

[0105] This preferred embodiment aims to construct an intelligent decision-making mechanism that can balance short-term recommendation performance with long-term system optimization. Its core objective is to introduce an ε-greedy exploration strategy to proactively explore patents that currently have low evaluation value but may possess potential value, while ensuring the overall performance of the recommendation system. This addresses the information cocoon effect and cold start problem commonly found in traditional recommendation systems, providing diverse training data for the system's continuous evolution.

[0106] This implementation introduces a stochastic exploration mechanism into the recommendation decision-making process, striking a balance between utilizing known optimal solutions and exploring unknown possibilities with a controllable probability. This achieves a balance between the short-term and long-term benefits of the recommendation system.

[0107] Parameter initialization: Set the exploration rate ε, typically between 0.05 and 0.15, representing the probability of the system conducting an exploration. Set the utilization rate 1-ε, representing the probability of the system selecting the current optimal recommendation.

[0108] In the stochastic decision-making process, before generating the final recommendation list each time, the system generates a uniformly distributed random number within the range [0,1]. The approach is as follows: when the random number > ε, the Top-N patent based on dynamic recommendation value ranking is directly used as the final recommendation list. This is the system's mainstream decision-making method, ensuring recommendation effectiveness.

[0109] When the random number is less than or equal to ε, the system initiates the exploration mechanism.

[0110] The exploration candidate set is constructed by excluding patents already in the recommendation list from the original candidate patent set. From the remaining patents, the top K% (e.g., top 30%) of patents ranked by dynamic recommendation value score are selected to form the exploration candidate set. This design ensures that the exploration process is conducted only within a relatively high-quality patent pool, avoiding the user experience risks associated with completely random exploration.

[0111] The process involves exploration and list updates, randomly and evenly selecting a patent from the candidate set. This selected patent replaces the patent with the lowest dynamic recommendation value score in the original recommendation list. The final recommendation list is then generated and displayed to the user.

[0112] Establish a complete exploration recording mechanism to provide data support for subsequent model optimization. The system records complete contextual information for each exploration action, including: Feature vectors and original Vi scores of the explored patents; The feature vector and Vi score of the replaced patent; Subsequent user interaction data. These records serve as special training samples for subsequent incremental model training.

[0113] Traditional recommendation systems often result in a large number of potentially high-quality patents never gaining exposure due to low initial evaluation. Furthermore, the models can only optimize within the existing data distribution, making it difficult to discover new value patterns. This implementation method achieves continuous innovation by introducing a controlled exploration mechanism.

[0114] This implementation method, through planned exploration, provides opportunities to showcase long-tail patents and emerging technologies, enriching the diversity of recommended content and avoiding homogenization of recommendation results. The user feedback data generated by the exploration process, especially feedback on patents where the model is uncertain, contains rich learning signals that can significantly improve the effectiveness of subsequent incremental training.

[0115] In a preferred embodiment of the present invention, the dynamic recommendation value score is adjusted in response to the user's real-time interaction behavior data stream, specifically as follows: Based on the real-time user interaction data stream, determine the real-time interaction type to adjust the dynamic recommendation value score; The adjusted dynamic recommendation value score will be used for incremental training of the dynamic value network.

[0116] This preferred embodiment aims to construct a self-evolving system capable of continuously learning and improving from real user feedback. Its core objective is to transform actual user interactions into a direct driving force for model optimization, thereby addressing the fundamental problem of traditional recommendation system models being rigid and unable to adapt to changes in user preferences, and achieving continuous improvement and long-term stability of the system's recommendation capabilities.

[0117] This implementation achieves continuous self-optimization of the system through refined feedback processing and an intelligent training mechanism. Similarly, the dynamic recommendation value score is adjusted based on the established precise mapping system from user behavior to model training signals.

[0118] The adjusted dynamic recommendation value score will be used for incremental training of the dynamic value network.

[0119] In real-world business environments, user preferences, market trends, and technological hotspots are constantly changing, making it difficult for static recommendation models to maintain excellent performance over the long term. This implementation method establishes a real-time feedback learning loop, enabling the system to keep pace with environmental changes. This is the core mechanism for maintaining the long-term competitiveness of the recommendation system.

[0120] The present invention also provides a computer-readable storage medium having a computer program stored thereon. The program implements the method when executed by the processor.

[0121] Therefore, any effect that can be achieved by the reinforcement learning-based intelligent patent recommendation method will not be elaborated here.

[0122] For any parts not mentioned in this invention, existing technologies can be used or referenced.

[0123] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0124] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A patent intelligent recommendation method based on reinforcement learning, characterized in that, include: Based on the collected patent data and combined with market dynamic data streams, the heat trend score and market capital attention score of the technology field are determined, and the real-time popularity factor is determined by combining user interaction behavior data streams. Based on user needs, a candidate patent set and basic semantic matching score are generated. Real-time market factors are obtained based on the technical field popularity trend score and market capital attention score. Dynamic recommendation value score is obtained based on the basic semantic matching score, the market factors, the real-time popularity factor, and the static quality score, and a recommendation list is generated. In response to the real-time user interaction data stream, the dynamic recommendation value score is adjusted for incremental training of the dynamic value network, and an update deployment is performed based on the training results.

2. The patent intelligent recommendation method based on reinforcement learning according to claim 1, characterized in that, The market dynamics data stream is as follows: The market dynamics data stream includes news data streams and investment and financing data streams; Based on the patent data and the news data stream, the frequency of keywords in the historical time series corresponding to each technical field is obtained. Through the time series prediction model, the predicted frequency value of keywords corresponding to the technical field is obtained. Based on the correspondence between the patent data and the technical field, the technical field popularity trend score of the patent data is obtained. Based on the patent data and the investment and financing data flow, and according to the historical financing amount, a market capital attention score is obtained.

3. The patent intelligent recommendation method based on reinforcement learning according to claim 1, characterized in that, By combining user interaction behavior data streams, the real-time popularity factor is determined as follows: Based on the user interaction behavior data stream, user interaction behavior types are divided, and factor scores are assigned to each type to obtain a daily popularity score. Based on the daily popularity score, a real-time popularity factor is obtained by applying a decay weighting through an exponential decay model, wherein the half-life of the exponential decay model is determined according to the period of the popularity score.

4. The patent intelligent recommendation method based on reinforcement learning according to claim 1, characterized in that, Based on the technology sector trend score and market capital attention score, real-time market factors are obtained, specifically: Based on the trend of technological focus and the level of market capital attention, a weighted average is used to obtain real-time market factors. The weighting is determined based on the user's industry and adjusted according to the technical field corresponding to the patent data.

5. The patent intelligent recommendation method based on reinforcement learning according to claim 1, characterized in that, Based on the basic semantic matching score, the market factor, the real-time popularity factor, and the static quality score, the dynamic recommendation value score is obtained, specifically as follows: Based on the basic semantic matching score, the market factor, the real-time popularity factor, and the static quality score, a multi-layer dynamic value network is input, wherein the multi-layer dynamic value network includes a content quality channel and a market popularity channel. Through the content quality channel, a first abstract feature representation is obtained based on the basic semantic matching score and the static quality score; through the market popularity channel, a second abstract feature representation is obtained based on the market factor and the real-time popularity factor. Based on the first abstract feature representation and the second abstract feature representation, weights are generated through an attention subnetwork, fused through a fully connected layer, and a dynamic recommendation value score is obtained through an activation function.

6. The patent intelligent recommendation method based on reinforcement learning according to claim 5, characterized in that, The content quality channel and the market popularity channel are specifically as follows: Both the content quality channel and the market popularity channel include at least two neural network layers. The number of neural network layers and / or neurons in the market popularity channel is greater than that in the content quality channel.

7. The patent intelligent recommendation method based on reinforcement learning according to claim 5, characterized in that, Based on the first abstract feature representation and the second abstract feature representation, weights are generated through an attention sub-network, specifically as follows: Based on the first abstract feature representation and the second abstract feature representation, the initial weights are obtained through the attention subnetwork; Based on user needs, the system perceives contextual information and makes adjustments according to the user's industry and the technical field corresponding to the patent data.

8. The patent intelligent recommendation method based on reinforcement learning according to claim 1, characterized in that, After generating the recommendation list, it also includes: Generate random numbers based on the dynamic recommendation value score; If the random number is greater than the preset exploration rate, then the final recommendation list is determined. Otherwise, from the patent data outside the recommendation list, a candidate set is selected based on the dynamic recommendation value score, and a patent data is randomly selected from the candidate set to replace the patent data with the lowest dynamic recommendation value score in the recommendation list.

9. The patent intelligent recommendation method based on reinforcement learning according to claim 1, characterized in that, In response to the real-time user interaction data stream, the dynamic recommendation value score is adjusted as follows: Based on the real-time user interaction data stream, determine the real-time interaction type to adjust the dynamic recommendation value score; The adjusted dynamic recommendation value score will be used for incremental training of the dynamic value network.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 9.

Images