Science and technology public text intelligent classification and service method and device based on deep learning

Abstract

The invention discloses a science and technology public text intelligent classification and service method and device based on deep learning. The method comprises the steps that multi-source science and technology public text data are acquired and preprocessed; extracting keywords by adopting a keyword extraction algorithm, and splicing the keywords with the public text title to form enhanced text features; constructing a multi-dimensional public text classification system and performing data annotation; performing feature extraction and fine adjustment by adopting a BERT pre-training model to obtain a classification model; automatically classifying the newly-added public texts and visually presenting the newly-added public texts; and generating a personalized recommendation result based on the user portrait and the public text feature index. The invention further relates to a technical scheme of multi-objective quality diversity optimization, heterogeneous resource allocation and fusion of the LPLC2 neural network and the BERT. The technical problems that a traditional method is limited in complex semantic understanding ability, single in classification dimension and lack of an integrated solution are solved, and the accuracy of science and technology public text classification and the intelligent level of service are improved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing and deep learning technology, and in particular to a method and device for intelligent classification and service of scientific and technological public texts based on deep learning. Background Art

[0002] Text classification and intelligent services for scientific and technological public documents are important areas of focus for public service management and corporate innovation. With the increasing number of scientific and technological public documents and their increasing complexity, how to efficiently and accurately classify public documents and provide intelligent services has become a pressing technical issue.

[0003] Currently, common technologies for classifying scientific and technological public text primarily rely on keyword matching and traditional machine learning methods. For example, statistical methods such as TF-IDF are used to extract text features, combined with classifiers such as SVM and random forests to build models, or rule-based methods are used to classify and serve public text. These methods can achieve certain results in specific scenarios, but their ability to understand complex semantics is limited.

[0004] With the development of natural language processing technology, pre-trained language models have been introduced into the field of public text classification, capturing the semantic features of public text through deep neural network architectures. This type of method utilizes the contextual understanding capabilities of language models to better grasp the semantic connotations of public text, thereby improving classification accuracy. However, existing technologies have several significant problems in the public text classification process: first, insufficient consideration of the structural characteristics of public text, making it difficult to accurately extract key information; second, the classification dimensions are relatively single and cannot meet the needs of multi-dimensional public text analysis; and third, there is a lack of a complete integrated solution from data processing to intelligent services. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and device for intelligent classification and service of scientific and technological public texts based on deep learning, aiming to solve the technical problems in the existing technology of insufficient consideration of the structural characteristics of public texts, single classification dimensions and lack of integrated solutions.

[0006] To achieve the above objectives, the present invention provides the following technical solutions: A deep learning-based intelligent classification and service method for scientific and technological public texts, including: Acquire multi-source science and technology public text data, perform text cleaning, deduplication and format unification processing on the multi-source science and technology public text data, and obtain a pre-processed multi-source science and technology public text data set; Based on the preprocessed multi-source science and technology public text dataset, a keyword extraction algorithm is used to extract keywords and splice them with public text titles to form enhanced text features; Based on the enhanced text features, a multi-dimensional public text classification system including talent support, financial support, technical support, and regulatory control is constructed, and data annotation is performed to establish a training data set; The BERT pre-trained model is used to perform feature extraction and domain adaptability fine-tuning on the training dataset, and a common text classification model is obtained through parameter optimization and ensemble learning; Automatically classify the newly added multi-source scientific and technological public text data to be classified based on the public text classification model to obtain classification results; and use data mining technology to analyze the distribution characteristics of public texts, and generate analysis result charts through visualization tools; Based on user interest data including user registration information, browsing history and search keyword data, a user portrait is constructed; and based on the classification results and the enhanced text features, a public text feature index is constructed; and based on the user portrait and the public text feature index, a recommendation algorithm is used to calculate the similarity between user interests and public text features to generate personalized public text recommendation results.

[0007] Preferably, the multi-source science and technology public text data is subjected to text cleaning, deduplication and format unification processing to obtain a pre-processed multi-source science and technology public text data set, including: Utilize an automated data collection framework to collect data from websites at all levels, call API interfaces to obtain structured public text information, and form an original multi-source science and technology public text dataset containing title, document number, release date, and text content; The original multi-source science and technology public text dataset is subjected to HTML tag removal and format normalization processing, and duplicate public texts are removed by a hash algorithm to obtain the preprocessed multi-source science and technology public text dataset.

[0008] Preferably, the method further includes performing structured analysis on the preprocessed multi-source science and technology public text dataset: Based on the characteristics of public text formats, the preprocessed multi-source science and technology public text dataset is decomposed into title, text, and attachment parts, and key information including public text goals, specific measures, and implementation time are extracted from the text to form structured multi-source science and technology public text data; The structured multi-source scientific and technological public text data is subjected to terminology normalization and format unification processing to obtain a standardized multi-source scientific and technological public text data set.

[0009] Preferably, the method of extracting keywords using a keyword extraction algorithm and combining them with public text titles to form enhanced text features includes: Obtain a science and technology public text domain dictionary, and use a Chinese word segmentation tool in combination with the science and technology public text domain dictionary to perform word segmentation processing on the preprocessed multi-source science and technology public text dataset to obtain a public text word segmentation result set; Extracting a set of keywords representing the core content of the text from the public text word segmentation result set based on the TextRank algorithm; The keyword set is concatenated with the title of the preprocessed multi-source science and technology public text dataset, and converted into a vector representation using word embedding technology to obtain the enhanced text feature.

[0010] Preferably, the construction includes a multi-dimensional public text classification system with talent support, financial support, technical support, and regulatory control, and data annotation, and the establishment of a training data set, including: Design the multi-dimensional public text classification system including the public text tool dimension, the public text field dimension, and the public text level dimension, and formulate classification standards and judgment rules; A stratified sampling method is adopted to select public text samples, and domain experts are organized to manually annotate according to the classification standard and the judgment rule. The annotation quality is ensured through cross-annotation by multiple people to obtain the training data set.

[0011] Preferably, the training dataset is subjected to feature extraction and domain adaptability fine-tuning using a BERT pre-training model, and a common text classification model is obtained through parameter optimization and ensemble learning, including: Dividing the training data set into a training set and a test set, performing BERT word segmentation processing on the text data of the training data set, and expanding the training samples through data augmentation operations; Fine-tune the Chinese BERT model for domain adaptability, setting parameters including learning rate and batch size, and using the cross-entropy loss function and Adam optimizer for model training. An ensemble learning method is used to combine the prediction results of multiple models to improve classification accuracy and obtain the common text classification model.

[0012] Preferably, the BERT pre-training model is used for feature extraction and domain adaptation fine-tuning, including: Based on the preprocessed multi-source science and technology public text dataset, a multi-dimensional feature space including semantic relevance, time relevance, and institutional relevance is constructed, and semantic vectors are calculated and the influence of public texts is evaluated using a word embedding method; An incremental learning strategy is adopted to enable the public text classification model to continuously learn new public text information, an anomaly detection algorithm is used to identify emerging concepts to dynamically expand the feature space, and a public text knowledge graph is established to achieve automatic expansion of concept boundaries; Multiple optimization objectives including classification accuracy, category balance, and diversity are defined, and a multi-objective optimization algorithm is used to find the balance point of each objective to obtain a public text classification model with multi-objective optimization.

[0013] Preferably, the multi-objective quality diversity optimization further includes: A diversity balance strategy is introduced to ensure that the personalized public text recommendation results cover different types of public text information, and a diversity reward mechanism is designed to encourage the multi-objective optimized public text classification model to explore public text areas that users have not been exposed to; An attention mechanism is used to dynamically evaluate the importance weight of each feature in the multi-dimensional feature space, and a feature feedback loop is established to adjust the feature weight system through user interaction data; Expand traditional evaluation indicators and introduce diversity indicators including coverage and novelty to build a diversity-aware evaluation system.

[0014] Preferably, the acquisition of multi-source scientific and technological public text data includes ensuring data acquisition stability through heterogeneous resource allocation, including: Develop a data collection engine that supports multiple protocols and is compatible with network protocols including HTTP, HTTPS, and FTP. Implement differentiated access control strategies to dynamically adjust collection parameters based on the characteristics of different data sources, build a distributed proxy pool, and obtain stable multi-source scientific and technological public text data collection capabilities. A microservices architecture is established based on container technology. By real-time monitoring and collecting resource usage, a priority task scheduling algorithm is designed to ensure that key data collection tasks are prioritized, thus achieving resource-aware dynamic scheduling. Build a dependency graph for data collection services, apply graph optimization algorithms to calculate the optimal service deployment method, establish edge nodes for data preprocessing, and obtain an optimized multi-source scientific and technological public text data collection architecture.

[0015] Preferably, the heterogeneous resource allocation further includes quality of service assurance for data transmission and processing, including: The design includes multi-level service quality definitions, including high-priority real-time collection, standard collection, and background batch processing, enabling dynamic perception of network conditions. Adaptive data compression and transmission strategies are adopted according to different QoS levels to ensure the stability and integrity of data collection. Establish a terminal device feature database, develop an adaptive data format conversion engine, implement data processing optimization based on network conditions, and design a progressive data loading strategy to ensure data acquisition efficiency in different network environments; Deploy a distributed data collection and monitoring system, design a data collection performance monitoring panel, implement an intelligent alarm mechanism to automatically trigger the repair process when the data collection quality falls below the threshold, and build an end-to-end data collection quality assurance system.

[0016] Preferably, the method further comprises: The BERT pre-trained model is used for feature extraction and domain adaptation fine-tuning, and the LPLC2 neural network is integrated to achieve multi-target detection of public text content, including: Divide the preprocessed multi-source technology public text dataset into text blocks according to sentences and paragraphs, convert the text blocks into high-dimensional vectors using BERT word embedding technology, map them to two-dimensional space through a dimensionality reduction algorithm to retain semantic similarity, construct a two-dimensional semantic feature map, and realize two-dimensional representation conversion of public text; Define key element types including public text objectives, implementation entities, support measures, and constraints; design multi-element annotation specifications; develop auxiliary annotation tools to support the annotation of text areas; and establish a fine-grained public text multi-element annotation system; The BERT model is used to extract the contextual semantic representation of public text. The BERT hidden layer features are converted into two-dimensional feature maps as the input of the LPLC2 network. The local pyramid and contrast modules of LPLC2 are applied to enhance the recognition of key element boundaries. A multi-task learning head is designed to simultaneously output the overall classification and element detection results, and the LPLC2-BERT fusion public text classification model is obtained.

[0017] Preferably, the LPLC2-BERT fused public text classification model further includes attention mechanism enhancement and joint optimization, including: A self-attention mechanism is introduced in the BERT encoding layer to capture long-range dependencies between common text words. A cross-modal attention module is designed to enable the LPLC2 network to focus on key areas of the feature map based on semantic importance. This combines channel attention and spatial attention to enhance feature representation. A hierarchical attention fusion mechanism is constructed to integrate features at different levels to achieve attention-enhanced feature extraction. Construct a joint loss function that includes a common text classification loss and an element detection loss, introduce the consistency loss between BERT representation and LPLC2 features, and design a dynamic weight balancing strategy to adjust the contribution ratio of each loss to achieve multi-task joint optimization training; Implement model quantization and pruning to reduce inference time, develop a non-maximum suppression algorithm to filter redundant detection results, construct a logical association relationship diagram between detection elements, and obtain a structured public text understanding representation and an optimized public text classification model.

[0018] The present invention also provides a deep learning-based intelligent classification and service device for scientific and technological public texts, including: A data collection and preprocessing module is used to obtain multi-source scientific and technological public text data, and perform text cleaning, deduplication and format unification processing on the multi-source scientific and technological public text data to obtain a preprocessed multi-source scientific and technological public text data set; A feature extraction enhancement module is used to extract keywords based on the preprocessed multi-source science and technology public text dataset using a keyword extraction algorithm and combine them with the public text titles to form enhanced text features; A classification system construction module is used to build a multi-dimensional public text classification system including talent support, financial support, technical support, and legal control based on the enhanced text features, and to perform data annotation and establish a training data set; A model training module is used to perform feature extraction and domain adaptability fine-tuning on the training dataset using the BERT pre-trained model, and obtain a common text classification model through parameter optimization and ensemble learning; An intelligent analysis and visualization module is used to automatically classify the newly added multi-source scientific and technological public text data to be classified based on the public text classification model to obtain classification results; and use data mining technology to analyze the distribution characteristics of public texts and generate analysis result charts through visualization tools; The recommendation service module is used to build a user portrait based on user interest data including user registration information, browsing history and search keyword data; and to build a public text feature index based on the classification results and the enhanced text features; and based on the user portrait and the public text feature index, use a recommendation algorithm to calculate the similarity between user interests and public text features to generate personalized public text recommendation results.

[0019] The beneficial effects of the present invention are: 1. This paper uses multi-source public text data collection and structured parsing technology to design an automated data collection program and API call method for public service department websites at different levels. It also uses a structured parsing algorithm to extract key components of public text, achieving standardized processing of public text data. 2. The public text feature enhancement method proposed in this paper, through the construction of a domain dictionary and a specific keyword extraction algorithm, combines public text titles and keywords to form enhanced text features, effectively improving the subsequent classification model's ability to understand public text; 3. This invention innovatively designs a science and technology public text classification system that includes multiple dimensions such as public text tools, public text fields, and public text levels, and formulates detailed classification standards and judgment rules, laying the foundation for the accurate classification of public texts; 4. This paper uses the BERT pre-training model to fine-tune domain adaptability based on the characteristics of scientific and technological public texts, and improves classification accuracy through ensemble learning methods, thus solving the problem of semantic understanding of public texts. 5. This invention constructs a public text association network based on the citation relationship and topic similarity of public texts, identifies core public text nodes and public text clusters through social network analysis methods, and reveals the complex relationships and evolution paths between public texts; 6. The present invention combines user portraits and public text feature representations to design a public text recommendation algorithm that takes into account both timeliness and relevance, thereby realizing accurate science and technology public text information push services to meet the personalized needs of different users. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 This is a flow chart of the method for intelligent classification and service of scientific and technological public texts based on deep learning of the present invention; Figure 2 This is a structural diagram of the scientific and technological public text intelligent classification and service device based on deep learning of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0023] like Figure 1 As shown, the deep learning-based intelligent classification and service method for scientific and technological public texts provided by the present invention includes the following steps: Step S1: Acquire multi-source science and technology public text data, perform text cleaning, deduplication and format unification on the multi-source science and technology public text data, and obtain a pre-processed multi-source science and technology public text data set. This step is the basic link of the entire science and technology public text intelligent classification and service method. First, science and technology public text data is acquired through multiple channels, including official websites of public service departments at all levels, professional databases and open API interfaces, etc. The acquired raw data usually contains a lot of noise and redundant information, so it is necessary to conduct text cleaning processing, remove interference elements such as HTML tags and special symbols, and normalize the text format. At the same time, the hash algorithm is used to perform fingerprint calculation on the text content, identify and remove duplicate public texts, and ensure the uniqueness and quality of the data set. In addition, it is also necessary to unify the formats of data from different sources, including character encoding unification, time format standardization and text structure normalization, etc., to finally form a pre-processed science and technology public text data set with a clear structure and unified format, providing a high-quality data foundation for subsequent feature extraction and classification model training.

[0024] Step S2: Based on the preprocessed multi-source science and technology public text dataset, a keyword extraction algorithm is used to extract keywords and concatenate them with the public text titles to form enhanced text features. After data preprocessing, this step aims to extract key features from the public text that represent its core content. First, a specially constructed science and technology public text dictionary and Chinese word segmentation tools are used to accurately segment the preprocessed text, identifying professional terms and important concepts. Then, algorithms such as TextRank are applied to extract a set of keywords from the segmentation results that best represent the core content of the text. These keywords typically summarize the text's theme and key points. Next, the extracted keyword set is concatenated with the public text title to form an enhanced text representation. This concatenation strategy simultaneously preserves the general information in the title and the detailed content in the text, making the text features more comprehensive and rich. Finally, word embedding technologies such as Word2Vec are used to convert these text features into vector representations. Combined with metadata features of the public text (such as publication time and issuing institution), a multi-dimensional feature representation is constructed. This provides richer input information for the subsequent classification model and improves the model's understanding of the public text's semantics.

[0025] Step S3: Based on the enhanced text features, a multidimensional public text classification system is constructed, including talent support, financial support, technical support, and regulatory support, and data annotation is performed to establish a training dataset. This step is the core step in the classification of science and technology public texts. First, based on the content characteristics and actual application needs of science and technology public texts, a multidimensional classification system is designed. This includes public text tool dimensions (such as talent support, financial support, technical support, and regulatory support), public text domain dimensions (such as artificial intelligence, biomedicine, and new materials), and public text level dimensions (such as national, provincial, and municipal). This multidimensional classification system comprehensively describes the characteristics and uses of science and technology public texts from different perspectives. After the classification system is established, detailed classification standards and judgment rules are formulated to clarify the definition, scope, and judgment boundaries of each category to ensure consistency and accuracy in the annotation process. Next, a stratified sampling method is used to select representative public text samples from the preprocessed dataset, and domain experts are organized to manually annotate according to established standards. To ensure annotation quality, a multi-person cross-annotation process is used. Annotation quality is evaluated by calculating annotation consistency indicators, and controversial samples are discussed and revised. Through this series of rigorous labeling processes, a high-quality training dataset was finally established, providing reliable supervision signals for subsequent model training.

[0026] Step S4: The BERT pre-trained model is used to perform feature extraction and domain adaptability fine-tuning on the training dataset. Through parameter optimization and ensemble learning, a public text classification model is obtained. This step uses deep learning technology to construct a science and technology public text classification model. First, the labeled training dataset is divided into a training set and a test set according to a certain ratio, and the text data is pre-processed using the BERT word segmenter and converted into the input format required by the model. At the same time, the training samples are expanded through data augmentation techniques such as synonym replacement and back translation to improve the generalization ability of the model. Then, domain adaptability fine-tuning is performed based on the pre-trained Chinese BERT model, and the model parameters are adjusted according to the characteristics of science and technology public text, including the optimization of hyperparameters such as learning rate, batch size, and training rounds. During the training process, the cross-entropy loss function is used to evaluate the gap between the model prediction and the true label, and the Adam optimizer is used to update the model parameters. At the same time, strategies such as learning rate decay are applied to avoid overfitting. After training is completed, the test set is used to evaluate the model performance, and indicators such as accuracy, precision, and recall are calculated. The model is then tuned based on the evaluation results. Finally, ensemble learning methods, such as voting and stacking, are used to combine the prediction results of multiple basic models to further improve classification accuracy. Ultimately, a public text classification model with excellent performance is obtained, which is deployed as a Web service and provides an API interface for other calls.

[0027] Step S5: Automatically classify the newly added multi-source science and technology public text data based on the public text classification model to obtain classification results. Data mining techniques are then used to analyze the distribution characteristics of the public text, and visualization tools are used to generate analysis charts. After model training is complete, this step applies the trained classification model to real-world scenarios to automatically classify the newly added science and technology public text data. The newly added data undergoes the same preprocessing and feature extraction as the training data. The processed data is then input into the classification model to obtain classification prediction results and their confidence levels. Samples with low confidence levels are marked as requiring manual review to ensure the reliability of the classification results. After classification, data mining techniques are applied to the classification results for multi-dimensional analysis, including temporal analysis (e.g., the publication trends and cyclical characteristics of various types of public texts), regional analysis (e.g., differences in the distribution of public texts across regions), and public text type analysis (e.g., the proportion and changes of various support policies). These analyses can identify hot spots and underserved areas in public texts, providing data support for decision makers. Finally, using visualization tools such as Echarts and D3.js, the analysis results are presented in the form of intuitive charts, including public text distribution heat maps, time evolution trend charts, association network diagrams, etc., and interactive data panels are designed to support users in multi-dimensional data screening and in-depth analysis, and generate comprehensive public text analysis visualization reports.

[0028] Step S6: Based on user interest data, including user registration information, browsing history, and search keyword data, a user profile is constructed. A public text feature index is constructed based on the classification results and the enhanced text features. Based on the user profile and the public text feature index, a recommendation algorithm is used to calculate the similarity between the user's interests and public text features, generating personalized public text recommendation results. This step implements an intelligent recommendation service for scientific and technological public texts. First, a multi-dimensional user interest profile is constructed by analyzing the user's registration information (such as industry, position, research field, etc.), browsing history (such as the types and content of public texts viewed), and search keyword data (reflecting the user's immediate interests). These profiles comprehensively reflect the user's preferences and needs for scientific and technological public texts. Simultaneously, based on the classification results and enhanced text features from the previous steps, a feature vector is constructed for each public text, and an efficient feature index is established to support fast retrieval and matching. Once the user profile and public text feature index are in place, algorithms such as collaborative filtering and content-based recommendation are used to calculate the similarity between the user's interest features and public text features, generating a preliminary recommendation list. The recommended results are then ranked and optimized based on factors such as timeliness (prioritizing newly released, relevant public documents) and relevance (matching with the user's current areas of interest), ultimately generating personalized public document recommendations. This intelligent recommendation service helps users quickly find the most relevant content within the vast amount of scientific and technological public documents, significantly improving the efficiency and accuracy of information acquisition and providing valuable decision-making support for researchers, business innovators, and policymakers.

[0029] In one embodiment of the present invention, step S1 performs text cleaning, deduplication, and format unification processing on the multi-source science and technology public text data to obtain a pre-processed multi-source science and technology public text data set, including: Step S1.1: Utilize an automated data collection framework to collect data from websites at all levels, invoking APIs to retrieve structured public text information, generating a dataset of original multi-source scientific and technological public text, including titles, document numbers, publication dates, and text content. This substep is the core work of the data collection phase. First, a Python-based automated data collection framework was developed, integrating tools such as Scrapy and Requests. Specialized crawlers were designed for the official websites of public service departments at the national, provincial, and municipal levels. These crawlers automatically identify website structures, locate public text publication pages, and extract key information according to pre-set rules. To account for structural differences and anti-crawler mechanisms across different websites, adaptive page parsing strategies and dynamic request parameter adjustments were implemented to ensure stable and comprehensive data collection. In addition to web crawling, structured public text data was directly acquired by invoking open APIs provided by various public service departments. These APIs typically provide more standardized and complete data formats, supplementing information that may be missed by web crawling. Through the combination of these two channels, a dataset of original multi-source scientific and technological public text, including key fields such as public text titles, document numbers, publication dates, issuing agencies, and text content, was generated. This dataset covers various types of public texts such as science and technology policies, funding support plans, talent introduction projects, etc. issued by public service departments at all levels, laying a solid foundation for subsequent data processing and analysis.

[0030] Step S1.2: HTML tags are removed and format normalized on the original multi-source science and technology public text dataset. Duplicate public text is removed using a hashing algorithm to obtain the preprocessed multi-source science and technology public text dataset. After obtaining the raw data, this substep performs data cleaning and preprocessing. First, regular expressions and tools such as BeautifulSoup are used to remove non-text elements such as HTML tags, JavaScript code, and CSS styles from the crawled text content, extracting the pure text content. The text is then format normalized, including unified character encoding (converting to UTF-8), normalizing whitespace (e.g., replacing extra spaces and line breaks), and correcting special symbols (e.g., converting full-width to half-width characters and standardizing Chinese and English punctuation). Regular expressions are also used to identify and correct common formatting errors, such as inconsistent date formats and confusing document number formats. To ensure the uniqueness and quality of the dataset, a SHA-256-based hashing algorithm is used to calculate a unique hash value for the core content of each public text (primarily the title and body), which serves as the text's "fingerprint." By comparing these hash values, we can efficiently identify and remove duplicate public text, including exact duplicates and highly similar content. This deduplication process is particularly important because the same public text may appear on multiple websites or in multiple versions, significantly improving the efficiency and accuracy of subsequent processing. After these cleaning and normalization steps, we ultimately obtained a pre-processed multi-source scientific and technological public text dataset with a unified format and unique content, providing high-quality data input for subsequent feature extraction and model training.

[0031] In one embodiment of the present invention, step S1 further includes performing structured analysis on the preprocessed multi-source science and technology public text dataset: Step S1.3: Based on the format characteristics of public texts, the preprocessed multi-source science and technology public text dataset is decomposed into title, main text, and attachment parts, and key information including public text goals, specific measures, and implementation time are extracted from the main text to form structured multi-source science and technology public text data. After completing basic text cleaning, this sub-step further performs structured analysis on the public text to mine its internal semantic structure and key information. First, based on the format characteristics of science and technology public texts, a special structured parsing algorithm is designed, which can accurately identify and separate the different components of the public text, including the title, main text, attachment list, etc. For the title part, the subject and type information of the public text are extracted; for the main text part, a more in-depth semantic analysis is performed to identify the paragraph structure and chapter division. Specifically, for the main text, an information extraction module based on a combination of rules and machine learning has been developed. This module can automatically identify and extract key information elements from public texts, including the main objectives of the public text (such as "promoting scientific and technological innovation" and "supporting industrial development"), specific support measures (such as funding amount, support methods, application conditions, etc.), implementation time (such as policy validity period, application deadline, etc.), and key information such as the responsible department. This extracted information is organized according to a predefined structured model, forming structured public text data with clear structure and clear semantics. This structuring process not only makes the content of public texts more organized and easier to understand, but also provides a more accurate information foundation for subsequent feature extraction, classification analysis, and intelligent recommendation, greatly improving the depth of understanding and application value of public text content. Step S1.4: The structured multi-source science and technology public text data is subjected to terminology normalization and format unification processing to obtain a standardized multi-source science and technology public text data set. After completing the structured analysis of the public text, this sub-step further standardizes the extracted structured data to ensure the consistency and comparability of the data. First, in response to the issue of terminology normalization, a standard terminology library in the field of science and technology public texts was constructed, which contains common policy terms, scientific and technological terms, and professional expressions, and a synonym mapping table was established to unify the same concept in different expressions into standard terms. For example, different expressions such as "artificial intelligence", "AI", and "intelligent technology" are unified into the standard term "artificial intelligence" to ensure the consistency of subsequent analysis. At the same time, automatic conversion between traditional and simplified Chinese is also realized, and all texts are unified into simplified Chinese format. In terms of format unification, time information was standardized, converting various date formats (such as "January 1, 2023," "2023.1.1," and "2023 / 01 / 01") into a unified standard format (such as "2023-01-01") to facilitate subsequent time series analysis. Similar standardization was also implemented for document number formats, unifying different formats (such as "Guofa

[2023] No. 1" and "Guofa

[2023] No. 1") into a standardized format. Furthermore, numerical representations were standardized, for example, converting different representations of "10 million," "10 million," and "10 million" into a standard numerical format. This series of terminology standardization and format unification ultimately resulted in a standardized scientific and technological public text dataset with consistent formatting and standardized terminology. This significantly improved the accuracy and efficiency of subsequent data analysis and provided a high-quality data foundation for model training and feature extraction.

[0032] In one embodiment of the present invention, obtaining multi-source scientific and technological public text data includes ensuring data collection stability through heterogeneous resource allocation, including: Step S1.5: Develop a data acquisition engine that supports multiple protocols and is compatible with network protocols including HTTP, HTTPS, and FTP, implement differentiated access control strategies to dynamically adjust acquisition parameters according to the characteristics of different data sources, build a distributed proxy pool, and obtain stable multi-source science and technology public text data acquisition capabilities. Specifically, for different public service department websites and data sources, the present invention has developed a set of data acquisition engines that can simultaneously support multiple network protocols. The engine not only supports standard HTTP and HTTPS protocols, but also supports file transfer protocols such as FTP, ensuring that science and technology public text data can be obtained from various data sources. At the same time, for the characteristics of different data sources, such as access frequency restrictions and response time characteristics, differentiated access control strategies are implemented, which can dynamically adjust parameters such as access frequency, concurrency, and timeout settings based on the real-time monitoring of data source responses. In addition, in order to avoid data collection interruptions caused by IP blocking, a distributed proxy IP pool is constructed, and data requests are made through intelligent rotation of proxy IPs, which effectively improves the stability and continuity of data collection and ensures the comprehensive acquisition of science and technology public text data.

[0033] Step S1.6: Build a microservices architecture based on container technology. Real-time monitoring and collection of resource usage are used to design a priority task scheduling algorithm to ensure that critical data collection tasks are prioritized, achieving resource-aware dynamic scheduling. In this embodiment, container technologies such as Docker are used to split data collection into multiple independent microservices, such as a web crawling service, an API call service, and a data cleansing service. Each service can be independently deployed and scaled. Monitoring tools such as Prometheus collect real-time resource usage information for each microservice, including CPU usage, memory usage, and network throughput. Based on this monitoring data, a priority task scheduling algorithm is designed. This algorithm allocates computing resources based on the importance and urgency of tasks, ensuring that critical data collection tasks (such as real-time updates of important public documents) receive priority resources and execution, while also appropriately delaying or limiting resources for non-critical tasks. This resource-aware dynamic scheduling mechanism ensures the smooth completion of important data collection tasks even with limited resources, improving overall resource utilization efficiency.

[0034] Step S1.7: Construct a dependency graph for data collection services, apply a graph optimization algorithm to calculate the optimal service deployment method, establish edge nodes for data preprocessing, and obtain an optimized multi-source science and technology public text data collection architecture. The present invention constructs a complete service dependency graph by analyzing the call relationships and data flows between various data collection services. The graph clearly shows the entire data flow process from the data source to the final storage. Based on this dependency graph, a graph optimization algorithm (such as the minimum spanning tree algorithm and the network flow algorithm) is applied to calculate the optimal solution for service deployment, reducing unnecessary network transmission and service jumps. At the same time, edge computing nodes are deployed near the data source. These nodes can perform preliminary data cleaning, format conversion, and deduplication operations at the first moment of data collection, significantly reducing the amount of data that needs to be transmitted to the central node. Through this optimized multi-source science and technology public text data collection architecture, high efficiency and scalability of data collection are achieved, which can cope with the ever-increasing amount of science and technology public text data and complex data source environments.

[0035] In one embodiment of the present invention, the heterogeneous resource allocation further includes quality of service assurance for data transmission and processing, including: Step S1.8: Design a multi-level Quality of Service (QoS) definition, including high-priority real-time collection, standard collection, and background batch processing. This allows for dynamic awareness of network conditions and the adoption of adaptive data compression and transmission strategies based on different QoS levels to ensure the stability and integrity of data collection. Based on the importance and timeliness of scientific and technological public text data, this invention designs a three-level Quality of Service (QoS) definition: high-priority real-time collection is used to process important and time-sensitive public texts, such as newly released national science and technology policies; standard collection is used to process routine scientific and technological public texts; and background batch processing is used to supplement and update historical public texts. Network probing technology monitors current network conditions in real time, including metrics such as bandwidth, latency, and packet loss rate. Based on these metrics and the task's QoS level, the most appropriate data compression algorithm and transmission strategy are dynamically selected. For example, when network conditions are good, high-priority tasks can use a lower compression algorithm to ensure data quality; when network conditions are congested, a higher compression algorithm can be used to ensure timely data transmission. Furthermore, breakpoint-resume transmission and data integrity verification mechanisms are implemented to ensure the integrity of data collection, preventing data loss or duplication, even in the event of a network outage.

[0036] Step S1.9: Establish a terminal device characteristics database, develop an adaptive data format conversion engine, implement data processing optimization based on network conditions, and design a progressive data loading strategy to ensure data acquisition efficiency in different network environments. To adapt to the characteristics and network environments of different terminal devices (such as servers, personal computers, and mobile devices), the present invention establishes a characteristics database containing information such as processing power, screen size, and network characteristics of various terminal devices. Based on this database, an adaptive data format conversion engine is developed that automatically adjusts the data format and size according to the characteristics of the terminal device. For example, for mobile devices with limited processing power, a more lightweight data format is generated; for high-performance servers, more detailed and complete data can be provided. Furthermore, data processing optimization based on network conditions is implemented. When network bandwidth is limited, critical data fields are prioritized and non-critical information is delayed. Furthermore, a progressive data loading strategy is designed, allowing users to view the loaded data before it is fully loaded, significantly improving user experience and data acquisition efficiency. These measures collectively ensure efficient acquisition and processing of scientific and technological public text data in various network environments.

[0037] Step S1.10: Deploy a distributed data collection and monitoring system, design a data collection performance monitoring panel, and implement an intelligent alarm mechanism to automatically trigger a remediation process when data collection quality falls below a threshold, thereby establishing an end-to-end data collection quality assurance system. To comprehensively monitor the quality and performance of the data collection process, the present invention deploys a distributed monitoring system that can track metrics such as the execution status, duration, resource consumption, and data quality of each data collection task in real time. Based on this monitoring data, an intuitive performance monitoring panel is designed, allowing administrators to quickly understand the overall system operation status and detailed information about each subtask. Furthermore, an intelligent alarm mechanism is implemented. When metrics such as the collection success rate, data integrity, or timeliness of a data source fall below a preset threshold, the corresponding remediation process is automatically triggered, such as adjusting collection parameters, switching proxy IP addresses, or notifying the administrator for manual intervention. Furthermore, comprehensive logging and analysis capabilities are established, enabling the mining of historical data to identify potential problem patterns and optimization opportunities. These measures together constitute an end-to-end data collection quality assurance system, ensuring that the collection process of public science and technology text data maintains high quality and reliability.

[0038] In one embodiment of the present invention, step S2 uses a keyword extraction algorithm to extract keywords and concatenate them with the common text title to form enhanced text features, including: Step S2.1: Obtain a domain dictionary for science and technology public text. Use a Chinese word segmentation tool in conjunction with the domain dictionary to segment the preprocessed multi-source science and technology public text dataset, obtaining a public text segmentation result set. This substep forms the foundation of the feature extraction process. First, a specialized domain dictionary for science and technology public text is constructed through various channels. This dictionary is constructed by integrating various sources: first, it collects and organizes professional terms and standard expressions in areas such as science and technology policy, funding support, and talent programs, including standardized expressions extracted from official documents; second, through statistical analysis of existing public text corpora, it automatically identifies high-frequency professional terms and unique expressions using methods such as TF-IDF and PMI, continuously enriching and updating the dictionary content. The resulting domain dictionary contains tens of thousands of professional terms, policy terms, and specific expressions in the science and technology public text field, providing important support for accurate word segmentation. During the word segmentation stage, mainstream Chinese word segmentation tools such as Jieba were used, and the constructed domain dictionary was imported as a custom dictionary, significantly improving the recognition accuracy of professional terms and policy expressions. During the word segmentation process, stop word filtering (removing common function words, conjunctions, and other words that contribute little to the text's theme) and part-of-speech tagging (identifying different parts of speech, such as nouns, verbs, and adjectives, to provide more information for subsequent feature extraction) were also performed. Furthermore, a synonym processing function was implemented, which can identify and merge similar professional terms. For example, synonyms such as "funding," "funding support," and "funding support" are unified to reduce redundancy in the feature space. Through this meticulous word segmentation process, a high-quality public text word segmentation result set is ultimately obtained. These results accurately reflect the semantic content and professional terminology distribution of public texts, laying a solid foundation for subsequent keyword extraction and feature construction.

[0039] Step S2.2: Extract a set of keywords representing the core content of the public text from the word segmentation results using the TextRank algorithm. After obtaining the word segmentation results, this substep aims to extract the keywords that best represent the core content of each public text. This substep primarily utilizes the TextRank algorithm, a graph-based algorithm inspired by Google's PageRank algorithm. This algorithm treats words in a text as nodes in a graph and the co-occurrence relationships between words as edges between nodes. It iteratively calculates the importance scores of the nodes to ultimately identify the most critical words in the text. Specifically, a word co-occurrence graph is first constructed based on the word segmentation results. A sliding window (typically 5 words) is set, and undirected edges are established between the words within the window. Each node in the graph is then assigned an initial weight, and the node weights are updated through multiple rounds of iterations until convergence or a preset number of iterations is reached. During this process, words with more co-occurrence relationships with important words receive higher weight scores. Finally, the words are ranked according to their weight scores, and the top-scoring words (typically 10-20) are selected as the keyword set for the public text. To further improve the accuracy of keyword extraction, the structural characteristics of public texts are also incorporated, assigning higher initial weights to words in important locations such as titles, abstracts, and chapter headings, allowing the algorithm to focus more on words in these locations. Parts of speech are also taken into account, prioritizing nouns, proper nouns, and other words that better represent the subject matter as keywords. By combining this combination of text structure and language characteristics, the TextRank algorithm can accurately extract the core keywords of each public text. This set of keywords highly summarizes the theme and key points of the public text, providing a refined semantic representation for subsequent feature construction.

[0040] The following key parameters are set in the specific implementation of the TextRank algorithm: First, the sliding window size is set to 5, establishing undirected edges between words within the window. For each word within the window, edge weights are calculated based on its distance within the window, with closer distances giving higher weights, with the inverse of the distance used as the weight. After constructing the word co-occurrence graph, each node is initially assigned a weight of 1.0, and node weights are then updated through iterations. The iterative calculations use the typical PageRank formula, which includes a damping coefficient (set to 0.85), the current node weight, and the sum of the weights contributed by neighboring nodes. The algorithm is considered converged when the maximum number of iterations is 100 or when the weight change between two consecutive iterations is less than 0.0001. After convergence, the nodes are ranked according to their final weights, and the 10-15 words with the highest weights are selected as keywords. The specific number is dynamically adjusted based on the length of the text and is typically calculated using a formula.

[0041] In addition, two improvements were made to the standard TextRank algorithm: first, the positional information of words was weighted, with the initial weight of words in the title, first paragraph, and last paragraph set to 1.5, while the initial weight of words in the text was 1.0; second, part-of-speech information was combined to give higher weights (multiplied by 1.2) to nouns, verbs, adjectives, and unregistered words, while the weight of parts of speech with less contribution, such as function words, was reduced (multiplied by 0.8).

[0042] Step S2.3: Concatenate the keyword set with the preprocessed titles of the multi-source science and technology public text dataset and convert them into vector representations using word embedding techniques to obtain the enhanced text features. This substep is a key step in feature construction. First, the keyword set extracted in the previous step is concatenated with the original titles of the public text to form the enhanced text representation. This concatenation strategy is based on two important considerations: first, the titles of public texts generally provide a high-level overview of the document's subject and purpose, encompassing the most core information; second, keywords extracted from the main text provide more detailed content points and technical terminology. By concatenating the two, the generality of the titles and the detail of the keywords are preserved, resulting in a more comprehensive text representation. After concatenation, the text is converted into a vector representation using advanced word embedding techniques. Specifically, a pretrained model such as Word2Vec, GloVe, or BERT is used to map each word into a dense vector of fixed dimension (typically 300 or 768 dimensions). All word vectors are then combined into a document vector using methods such as weighted averaging, which serves as the semantic representation of the public text. The importance of words is also taken into account during vector construction. For example, different weights are assigned to different words based on their TF-IDF values, ensuring that more recognizable words have a greater impact in the final vector. Furthermore, metadata features of public texts, such as publication time (converted into relative time features) and issuing institutions (converted into institutional type and level features), are incorporated into the text semantic vector to form a multi-dimensional feature representation. This enhanced feature representation, which integrates text semantics and metadata, comprehensively captures the content characteristics and contextual information of public texts, providing rich and effective feature input for subsequent classification models, significantly improving the model's understanding of public text semantics and classification accuracy.

[0043] In one embodiment of the present invention, step S3 constructs a multi-dimensional public text classification system including talent support, financial support, technical support, and regulatory control, performs data annotation, and establishes a training data set, including: Step S3.1: Design a multi-dimensional public text classification system, encompassing the public text tool dimension, the public text domain dimension, and the public text hierarchy dimension, and establish classification standards and judgment rules. This sub-step is the core work of constructing a classification system for science and technology public texts. Based on the analysis of a large number of science and technology public texts and the actual application needs, a classification system with three key dimensions was designed. First, the public text tool dimension reflects the methods and means of support provided by public texts, including talent support (such as talent introduction programs and scientific researcher incentive policies), financial support (such as scientific research funding and innovation project subsidies), technical support (such as technology transfer services and the opening of scientific research facilities), and regulatory control (such as industry standard setting and market access regulations). Second, the public text domain dimension describes the scientific and technological fields and industrial directions targeted by public texts, including multiple sub-sectors such as artificial intelligence, biomedicine, new materials, new energy, and information technology, as well as cross-disciplinary comprehensive policies. Third, the public text hierarchy dimension reflects the publisher and scope of application of public texts, including different administrative levels. Detailed classification standards and judgment rules have been developed for this classification system with three dimensions, including the definition, scope, typical characteristics, and boundary conditions of each category. For example, public documents related to funding support are required to include specific elements such as the funding amount, support method, and application conditions; while public documents in the field of artificial intelligence are required to explicitly mention key technologies or applications such as intelligent algorithms, machine learning, and computer vision. These classification standards and judgment rules form a complete set of classification guidance documents, providing a clear operational basis for subsequent data annotation, ensuring the consistency and accuracy of the annotation process, and providing end users with a clear framework for searching and filtering public documents.

[0044] Step S3.2: Public text samples were selected using a stratified sampling method. Domain experts were organized to manually annotate according to the classification criteria and judgment rules. The annotation quality was ensured through cross-analysis by multiple people, resulting in the training dataset. After completing the classification system design, this substep conducted preliminary data annotation. First, a stratified sampling method was used to select representative public text samples from the preprocessed dataset. Stratified sampling ensured a balanced distribution of samples across different time periods, publishing organizations, and content types, avoiding sample selection bias. Specifically, the dataset was divided into multiple strata based on the publication time, publishing organization, and preliminary content analysis of the public texts. Samples were then randomly selected from each stratum proportionally. Ultimately, approximately 5,000 representative public texts were selected as the annotated sample. Next, a multidisciplinary annotation team, including policy research experts, science and technology management personnel, and domain technical experts, was assembled to manually annotate the selected samples. The annotation process strictly adhered to the previously established classification criteria and judgment rules. Each annotator was required to select appropriate category labels for each public text along three dimensions and record the basis for their judgment. To ensure annotation quality, a multi-person cross-annotation process was employed: each public text was independently annotated by at least two annotators, and annotation consistency metrics (such as Cohen's Kappa coefficient) were calculated. For samples with high consistency, the majority annotation results were adopted; for samples with significant disagreement, expert discussions were held to reach consensus through in-depth analysis and discussion. Furthermore, regular annotation verification processes were implemented, with randomly selected annotated samples reviewed to promptly identify and correct potential annotation errors. This rigorous annotation process and quality control measures ultimately resulted in a high-quality training dataset, providing reliable supervision signals for subsequent model training and laying a solid foundation for the practical application of the classification system.

[0045] In one embodiment of the present invention, in step S4, the BERT pre-trained model is used to perform feature extraction and domain adaptability fine-tuning on the training dataset, and a common text classification model is obtained through parameter optimization and ensemble learning, including: Step S4.1: Divide the training dataset into a training set and a test set, perform BERT word segmentation on the text data of the training dataset, and expand the training samples through data augmentation operations. Before starting model training, this sub-step first scientifically divides and preprocesses the labeled training dataset. Using a stratified random sampling method, the dataset is divided into a training set and a test set in a ratio of 8:2 to ensure that the distribution of the two subsets in each category remains consistent, avoiding evaluation bias caused by uneven data distribution. The time factor is also taken into account during division to ensure that the test set contains newer public text samples to better evaluate the performance of the model when processing newly released public texts. After the data division is completed, the text data is processed using a BERT-specific word segmenter. The BERT word segmenter uses the WordPiece algorithm, which can decompose the text into subword units and effectively process uncommon words and compound words. It is particularly suitable for processing professional terms and policy statements in scientific and technological public texts. Token segmentation converts text into the input format required by the model, including token IDs (word numbers), attention masks (marking which parts of the text are valid), and segment IDs (used to distinguish different parts of the text). To enhance the model's generalization and robustness, various data augmentation operations were implemented. These include synonym replacement (randomly replacing words in the text using a synonym dictionary), back-translation (translating text into English and then back into Chinese, preserving semantics while modifying the presentation), and random insertions, deletions, and swaps (making subtle changes to the text structure while preserving semantics). These data augmentation techniques effectively expand the diversity of training samples, enabling the model to learn a richer range of linguistic expression variations and improving its ability to cope with a wide range of expressions. Data augmentation is particularly helpful in alleviating data imbalance in categories with fewer samples. This meticulous data preparation provides high-quality and diverse training data for subsequent model training, laying the foundation for the model's high performance and strong generalization.

[0046] Step S4.2: Fine-tune the Chinese BERT model for domain adaptability. Parameters including the learning rate and batch size are set, and the model is trained using the cross-entropy loss function and the Adam optimizer. This substep is the core of model training. A pretrained Chinese BERT model (such as bert-base-chinese) is used as the base model, and domain-adaptive fine-tuning is performed based on the characteristics of scientific and technological public text. The BERT model uses a Transformer architecture, which, through its self-attention mechanism, can capture long-range dependencies in text, making it particularly well-suited for processing the complex semantic structure found in scientific and technological public text. The model architecture design adds an output layer suitable for multi-label classification based on the BERT encoder. The model includes three independent classification heads, one for the tool dimension, one for the domain dimension, and one for the level dimension. To optimize model training, a series of key parameters are carefully configured. First, a small initial learning rate (typically 2e-5 to 5e-5) is used. This is combined with a learning rate warm-up and linear decay strategy, slowly increasing the learning rate at the beginning of training and then gradually decreasing it as training progresses to avoid training instability or getting stuck in local optima. Secondly, based on GPU memory limitations and training efficiency considerations, an appropriate batch size (typically 16 or 32) was set to maximize computing resource utilization while ensuring training stability. In addition, an appropriate number of training epochs (typically 3 to 5) was set, and an early stopping strategy was implemented, automatically stopping training when the validation set performance no longer improves over multiple epochs to prevent overfitting. In terms of loss function selection, the cross-entropy loss function was used for each classification dimension. It effectively measures the difference between the predicted distribution and the true label distribution and is particularly suitable for multi-category classification problems. For the optimizer, the Adam optimizer was selected. It combines the advantages of the momentum method and the adaptive learning rate method, automatically adjusting the learning rate of each parameter to accelerate convergence and improve training stability. Through these meticulous model designs and parameter optimizations, the BERT model was effectively transferred to the field of scientific and technological public text, enabling the model to accurately understand and classify various types of scientific and technological public text.

[0047] In terms of model architecture, this paper uses the BERT-base-chinese pre-trained model, which consists of 12 Transformer encoder layers, each with 12 attention heads and a hidden layer dimension of 768. The total number of model parameters is approximately 110 million. In the model structure design, three independent classification heads are added on top of the BERT encoder, responsible for classification tasks in the common text tool dimension, domain dimension, and hierarchical dimension respectively. Each classification head consists of a linear layer (768-dimensional input, number of categories output) and a softmax activation function.

[0048] The following hyperparameter settings were used: an initial learning rate of 3e-5, with a linear warmup strategy (1000 warmup steps) and a linear decay strategy; a batch size of 32; four training epochs, with an early stopping mechanism that automatically terminates training after two consecutive epochs of performance improvement on the validation set; a weight decay coefficient of 0.01 to mitigate overfitting; and a gradient clipping threshold of 1.0 to prevent exploding gradients. Training used the cross-entropy loss function and the Adam optimizer, with a first momentum coefficient of 0.9, a second momentum coefficient of 0.999, and a numerical stability constant of 1e-8.

[0049] To monitor the training process, we recorded the trends of training and validation loss. Typically, the training loss gradually decreased from an initial value of approximately 2.1 to around 0.3, while the validation loss decreased from an initial value of approximately 1.9 to around 0.4 before leveling off. Regarding performance metrics on the validation set, after model training, the average accuracy across all classification dimensions reached 92.7%, and the F1 score reached 91.3%, significantly outperforming traditional machine learning methods (such as SVM's 87.1% accuracy) and common deep learning methods (such as TextCNN's 89.4% accuracy).

[0050] Step S4.3: An ensemble learning method is used to combine the prediction results of multiple models to improve classification accuracy and obtain the common text classification model. After the base model is trained, this substep further improves classification performance using ensemble learning techniques. The core concept of ensemble learning is to combine the prediction results of multiple base models, pooling their knowledge to reduce the bias and variance of individual models and improve overall prediction accuracy and stability. Three main ensemble learning strategies are employed: voting, stacking, and model fusion. First, voting is the most straightforward ensemble method. Multiple BERT models with similar structures but different initializations or hyperparameters are trained. The classification results of each sample are then voted on, and the category with the most votes is selected as the final prediction. This method is simple and effective, reducing the random error of individual models. Second, stacking is a more complex ensemble method. It first trains multiple base models of different types, including pre-trained models such as BERT, RoBERTa, and ALBERT, as well as traditional deep learning models such as CNN and BiLSTM. The predictions of these base models are used as features and fed into a meta-learner (typically a logistic regression or lightweight neural network) for secondary learning to generate the final prediction. This method can effectively combine the advantages of different models and capture more complex patterns. Third, model fusion technology obtains a smoother and more reliable prediction distribution by weighted averaging the prediction probabilities of multiple models (rather than hard-voting category labels). Weights can be dynamically assigned based on the performance of each model on the validation set, so that models with better performance receive higher weights. Through the comprehensive application of these ensemble learning methods, the accuracy and robustness of classification have been significantly improved, especially when dealing with public texts with blurred boundaries or complex expressions. The final public text classification model not only achieved excellent performance indicators (such as accuracy, F1 value, etc.) on the test set, but also demonstrated good generalization ability and stability in practical applications. It can accurately classify various types of newly released scientific and technological public texts, providing a reliable foundation for subsequent analysis and recommendation services.

[0051] In one embodiment of the present invention, the feature extraction and domain adaptation fine-tuning using the BERT pre-training model include: Step S4.4: Based on the preprocessed multi-source science and technology public text dataset, a multidimensional feature space is constructed that includes semantic relevance, temporal relevance, and institutional relevance. Semantic vectors are calculated using word embedding methods to evaluate the influence of public texts. In this embodiment, not only the semantic content of public texts is considered, but also multidimensional features such as temporal and institutional features are integrated to construct a more comprehensive feature space. For semantic relevance, word embedding techniques such as Word2Vec and Doc2Vec are used to convert public text content into high-dimensional vector representations, capturing the text's semantic information. For temporal relevance, a time decay function is designed to give more weight to recently published public texts, reflecting their timeliness. For institutional relevance, an authority scoring system for issuing institutions is established, with different levels and types of public service departments and institutions assigned different weights. The combination of these multidimensional features enables a comprehensive assessment of the influence of public texts, providing a richer feature representation for subsequent classification and recommendation. Furthermore, a feature fusion mechanism is implemented that automatically learns the importance of features from different dimensions through an attention network and dynamically adjusts the weights of each dimension based on the specific task, making the feature representation more adaptable to different application scenarios.

[0052] Step S4.5: An incremental learning strategy is employed to enable the public text classification model to continuously learn new public text information. Emerging concepts are identified through anomaly detection algorithms, dynamically expanding the feature space and establishing a public text knowledge graph to automatically expand concept boundaries. Considering the constant updating and evolution of knowledge in the field of scientific and technological public text, the present invention employs an incremental learning strategy to enable the model to continuously learn newly emerging public text information without retraining. Specifically, newly released public text is regularly collected and compared with the prediction results of the existing model. Samples with low prediction confidence are marked as potential new concepts or categories. Anomaly detection algorithms (such as Isolation Forest and One-Class Support Vector Machine) are then used to further verify that these samples represent new knowledge points. Once confirmed, the feature space is dynamically expanded to create new representations for these new concepts. Simultaneously, a public text knowledge graph is constructed. Through entity recognition and relationship extraction, key concepts and their relationships in the public text are represented as a graph structure. This knowledge is then incorporated into the model using graph embedding techniques. The introduction of knowledge graphs enables the automatic expansion of concept boundaries and the capture of associations between concepts, greatly enhancing the model's ability to understand public texts in emerging fields and interdisciplinary disciplines.

[0053] Step S4.6: Define multiple optimization objectives, including classification accuracy, class balance, and diversity. A multi-objective optimization algorithm is used to find the balance between these objectives, resulting in a multi-objective optimized public text classification model. Traditional classification models typically focus solely on the single objective of classification accuracy. However, in practical applications, factors such as class balance and diversity are equally important. To this end, the present invention defines three key optimization objectives: classification accuracy (measured by metrics such as precision, recall, and F1 score), class balance (ensuring similar classification performance across classes to avoid neglecting certain classes), and diversity (ensuring the model can recognize and process diverse public text expressions). To simultaneously optimize these three potentially conflicting objectives, multi-objective optimization algorithms such as NSGA-II (Non-Dominated Sorting Genetic Algorithm II) and MOEA / D (Multi-Objective Evolutionary Algorithm with Decomposition) are employed. These algorithms use evolutionary computation to search for Pareto-optimal solutions in the objective space—i.e., solutions that cannot further improve other objectives without sacrificing one objective. From these Pareto-optimal solutions, the most suitable balance point is selected as the final model based on the needs of the specific application scenario. This multi-objective optimization method enables the public text classification model to maintain high accuracy while also being able to fairly process various types of public texts and adapt to diverse expressions, greatly improving the practicality and robustness of the model in practical applications.

[0054] In one embodiment of the present invention, the multi-objective quality diversity optimization further includes: Step S4.7: A diversity balancing strategy is introduced to ensure that the personalized public text recommendation results cover different types of public text information. A diversity reward mechanism is designed to encourage the multi-objective optimized public text classification model to explore public text areas that the user has not yet encountered. To avoid the common "information cocoon" problem in recommendation, the present invention introduces a diversity balancing strategy to ensure that the public text recommended to the user is not limited to their known areas of interest. Specifically, the Deterministic Point Process (DPP) model is employed, which naturally promotes diversity in the result set while ensuring relevance. A similarity matrix is ​​calculated between candidate public texts, and a DPP algorithm is used to select a subset of public texts that are both relevant and diverse as the final recommendation results. Furthermore, a diversity reward mechanism is designed. When the model recommends public text areas that the user has not previously encountered and the user provides positive feedback on these recommendations, the model is given additional rewards, encouraging it to continue exploring potential areas of interest. This mechanism is implemented through reinforcement learning, and the model gradually learns a balanced strategy for appropriately incorporating novel content while maintaining basic recommendation quality. Through these diversity optimization measures, we can provide users with more comprehensive and balanced public text recommendations, helping them expand their knowledge and discover new areas of public text interest.

[0055] Step S4.8: An attention mechanism is used to dynamically evaluate the importance weights of each feature in the multidimensional feature space. A feature feedback loop is established to adjust the feature weight system based on user interaction data. In a multidimensional feature space, the importance of different features for classification and recommendation often changes dynamically, depending on the specific task and user. To capture this dynamic nature, the present invention employs an attention mechanism that automatically adjusts the importance weights of each feature based on the current input and task. Specifically, a multi-head self-attention architecture is used, which allows the model to simultaneously learn relationships between features from multiple perspectives and assign appropriate weights to each feature. These weights are not fixed but are dynamically calculated based on the input public text content and user context, enabling the model to flexibly adjust feature importance for different situations. Furthermore, a feature feedback loop is established to continuously evaluate and adjust the feature weight system by collecting user interaction data with recommendation results (such as clicks, favorites, and shares). For example, if a user is found to be particularly interested in public text published by a specific organization, the weight of the institution-related feature will be increased; if a user tends to pay attention to the most recently published public text, the importance of the time-related feature will be increased. This feature weight adjustment mechanism based on user feedback enables continuous optimization of feature representation and provides more personalized and accurate public text classification and recommendation services.

[0056] Step S4.9: Expand traditional evaluation metrics by introducing diversity indicators, including coverage and novelty, to construct a diversity-aware evaluation system. Traditional recommendation evaluation focuses primarily on accuracy metrics, such as precision, recall, and F1 score. However, these metrics fail to fully reflect the actual effectiveness of recommendations, particularly in terms of diversity and user satisfaction. To this end, this invention expands the evaluation system by introducing a series of diversity-related metrics: coverage (measuring the proportion of public text categories covered by the recommendation results), novelty (measuring the proportion of content in the recommendation results that the user has not previously encountered), serendipity (measuring the proportion of content in the recommendation results that surprises the user), and diversity (measuring the degree of variation within the recommendation results). Together, these metrics form a diversity-aware evaluation system that can assess recommendation performance from multiple dimensions. Furthermore, a user satisfaction survey and A / B testing scheme are designed to further validate and optimize recommendation strategies by directly collecting user feedback and comparing the actual performance of different algorithms. Long-term, the invention will also track the expansion of user knowledge to evaluate the contribution of recommendations in promoting user understanding of a wider range of public text domains. This comprehensive evaluation system enables us to continuously optimize diversity and user experience while ensuring basic recommendation accuracy, providing a more balanced and valuable public text recommendation service.

[0057] In one embodiment of the present invention, the following steps are also included: The BERT pre-trained model is used for feature extraction and domain adaptation fine-tuning, and the LPLC2 neural network is integrated to achieve multi-target detection of public text content, including: Step S4.10: The preprocessed multi-source technology public text dataset is divided into text blocks by sentence and paragraph. The text blocks are converted into high-dimensional vectors using BERT word embedding technology. These vectors are then mapped to a two-dimensional space using a dimensionality reduction algorithm while preserving semantic similarity. A two-dimensional semantic feature graph is constructed to achieve a two-dimensional representation conversion of the public text. Traditional text processing methods typically treat text as a one-dimensional sequence, making it difficult to capture the complex structure and relationships within the text. To address this problem, the present invention proposes an innovative two-dimensional representation conversion method that converts public text documents into two-dimensional feature graphs similar to images, enabling the LPLC2 network, originally designed for image processing, to be applied to text analysis. Specifically, the public text is first divided into multiple text blocks based on natural paragraphs and sentences, with each block representing a semantic unit in the document. The BERT model is then used to encode each text block, resulting in a 768-dimensional high-dimensional vector representation that captures the deep semantic information of the text block. Next, these high-dimensional vectors are mapped to a two-dimensional space using a dimensionality reduction algorithm such as t-SNE or UMAP, while preserving the semantic similarity between the vectors. In this two-dimensional space, semantically similar text blocks are mapped to similar locations, forming semantic clusters. Finally, based on these two-dimensional coordinates, a two-dimensional semantic feature map is constructed, where each "pixel" corresponds to a text block, and its "color" (i.e., feature value) is determined by the semantic characteristics of that text block. This two-dimensional representation method not only preserves the semantic information of the text but also reflects the relative positions and relationships between text blocks, providing an ideal input format for subsequent multi-object detection.

[0058] Step S4.11: Define key element types, including public text objectives, implementation entities, support measures, and constraints. Design multi-element annotation specifications, develop auxiliary annotation tools to support the annotation of text regions, and establish a fine-grained public text multi-element annotation system. To enable refined analysis of public text content, this paper defines a series of key element types, including public text objectives (the problems or goals that the public text aims to solve), implementation entities (the institutions or organizations responsible for implementing the public text), support measures (the specific support methods and resources provided by the public text), and constraints (the restrictions and requirements for the implementation of the public text). Based on these element types, a detailed multi-element annotation specification is designed, clarifying the criteria, boundary definitions, and annotation formats for each type of element. To improve annotation efficiency and accuracy, a dedicated auxiliary annotation tool has been developed. This tool provides a visual interface that allows annotators to directly select regions on the text and assign category labels, while also providing real-time annotation suggestions and consistency checks. During the annotation process, a multi-person cross-annotation approach is adopted, whereby multiple annotators independently annotate the same document. Annotation quality is then assessed by calculating annotation consistency metrics (such as Cohen's Kappa coefficient). Samples with low consistency are discussed and corrected. Through this fine-grained multi-element annotation system, high-quality training data can be obtained, providing reliable supervision signals for subsequent multi-target detection models, and achieving accurate understanding and positioning of public text content.

[0059] Step S4.12: Use the BERT model to extract contextual semantic representations of public text. The BERT hidden layer features are converted into two-dimensional feature maps as input to the LPLC2 network. The LPLC2's local pyramid and contrast modules are applied to enhance key element boundary identification. A multi-task learning head is designed to simultaneously output overall classification and element detection results, resulting in an LPLC2-BERT fusion public text classification model. This innovative fusion of the BERT model and the LPLC2 network leverages the strengths of both models to achieve both overall classification of public text and accurate detection of internal elements. In terms of the model architecture, the BERT model is first used to process the raw text input to extract rich contextual semantic representations. Then, the hidden layer features of each BERT layer (particularly those in the intermediate layers, which contain rich grammatical and semantic information) are reorganized into two-dimensional feature maps as input to the LPLC2 network. The LPLC2 network is a neural network architecture designed specifically for object detection. Its core components include a local pyramid module (capturing multi-scale features) and a local contrast module (enhancing object boundaries). These modules effectively process the two-dimensional feature maps, identifying salient regions and boundaries within them, and thus accurately localizing key elements within public text. At the network's output, a multi-task learning head is designed to simultaneously perform two tasks: first, classifying the entire public text and outputting the public text category to which it belongs; second, detecting and locating key elements in the text, outputting each element's type and location information (coordinates on a two-dimensional feature map). This LPLC2-BERT fusion model, through end-to-end joint training, simultaneously optimizes classification and detection performance, achieving a comprehensive understanding and refined analysis of public text content.

[0060] The specific architecture of the LPLC2-BERT fusion network is as follows: First, the BERT encoder uses the standard BERT-Base-Chinese model (12-layer Transformer, 12 attention heads, 768-dimensional hidden layers), but only retains the first 8 layers to extract basic semantic features. This reduces computational complexity while preserving sufficient semantic information. The feature dimension after BERT encoding is a 768-dimensional vector multiplied by the sequence length.

[0061] In order to convert BERT's one-dimensional sequence features into the two-dimensional feature maps required by LPLC2, a feature reorganization module is designed. The module adopts the following steps: first, the hidden state of the last layer of BERT is rearranged into an approximately square two-dimensional feature map. The dimension becomes a four-dimensional tensor of batch size, height, width and number of channels, where the height and width are determined by the square root of the sequence length; then, the channel dimension is reduced from 768 to 256 through 1×1 convolution, resulting in a feature map with 256 channels.

[0062] The specific structure of the LPLC2 network includes: 1. Local Pyramid Module: This module consists of three parallel residual branches, each containing convolution kernels of different sizes (3×3, 5×5, and 7×7) to capture features at different scales. Each branch includes a convolutional layer, a batch normalization layer, and a ReLU activation function, which are then fused through addition and residually connected to the input features.

[0063] 2. Local Contrast Module: First, average pooling and max pooling are applied to the input features to obtain the channel average and maximum, respectively. These two features are then concatenated and subjected to dimensionality reduction through convolution. A sigmoid function is then used to generate a spatial attention map. Finally, the attention map is multiplied with the original features to enhance the contrast of salient areas.

[0064] 3. Feature enhancement path: This path consists of four groups of alternating local pyramid modules and local contrast modules, forming a deep feature extraction network. Each group is followed by a feature downsampling layer (2×2 max pooling) to expand the receptive field.

[0065] 4. Feature Pyramid Network: This network uses a top-down feature fusion approach to integrate feature maps of different scales. The specific structure includes a top-down upsampling path (using 2× upsampling) and lateral connections (using 1×1 convolution for channel alignment). The network outputs feature maps of four different scales, with the resolution halved sequentially.

[0066] 5. Multi-task learning head: contains two parallel task branches: a) Classification branch: The feature map is converted into a vector through global average pooling, and then the common text classification result is output through a fully connected layer; b) Detection branch: Apply convolution to the feature maps at each scale, and output the target position prediction (4 coordinate values) and category prediction (number of categories plus 1, including background class).

[0067] The total number of parameters in this fusion network is approximately 140 million, of which approximately 90 million are for BERT and approximately 50 million for LPLC2. The forward inference time on a GPU (such as a Tesla V100) is approximately 120 milliseconds per sample.

[0068] In one embodiment of the present invention, the LPLC2-BERT fusion public text classification model further includes attention mechanism enhancement and joint optimization, including: Step S4.13: A self-attention mechanism is introduced into the BERT encoding layer to capture long-range dependencies between words in public text. A cross-modal attention module is designed to enable the LPLC2 network to focus on key regions of the feature map based on semantic importance. This combines channel attention and spatial attention to enhance feature representation. A hierarchical attention fusion mechanism is constructed to integrate features from different levels, achieving attention-enhanced feature extraction. To further enhance the model's feature extraction capabilities, this paper introduces a multi-level attention mechanism into the LPLC2-BERT fusion model. First, the existing self-attention mechanism is strengthened in the BERT encoding layer, enabling the model to better capture long-range dependencies between words in public text, which is particularly important for understanding complex public text language. Second, an innovative cross-modal attention module is designed. This module can use the semantic information extracted by BERT as guidance to help the LPLC2 network focus on key regions in the feature map. Specifically, BERT's semantic representation is used to generate an attention weight map, which is then applied to the LPLC2 feature map to achieve semantically guided visual attention. Furthermore, the model combines channel attention (emphasizing important feature channels) and spatial attention (emphasizing important spatial locations). Through the Squeeze-and-Excitation network and spatial attention module, the model enhances the discriminability of feature representations in the channel and spatial dimensions, respectively. Finally, a hierarchical attention fusion mechanism is constructed to weightedly fuse features from different layers of BERT and LPLC2 to form a multi-scale, multi-level comprehensive feature representation. This multi-level attention enhancement mechanism enables the model to more accurately identify key information and important elements in public text, significantly improving the quality and efficiency of feature extraction.

[0069] Step S4.14: Construct a joint loss function consisting of a common text classification loss and an element detection loss. This loss introduces a consistency loss between the BERT representation and the LPLC2 features, and a dynamic weight balancing strategy is designed to adjust the contribution of each loss, enabling multi-task joint optimization training. To effectively train the LPLC2-BERT fusion model, this paper designs a complex joint loss function consisting of multiple components that jointly guide the model's optimization direction. First, the common text classification loss uses a cross-entropy loss function to evaluate the model's prediction accuracy for the overall common text category. Second, the element detection loss consists of two parts: a bounding box regression loss (using a smoothed L1 loss to measure the difference between the predicted and true bounds) and an element category classification loss (using a cross-entropy loss to evaluate the accuracy of element type prediction). Furthermore, to ensure consistency between the semantic features extracted by BERT and the visual features processed by LPLC2, a feature consistency loss is introduced. This minimizes the distance between the two feature representations (such as cosine distance or KL divergence), promoting information alignment and complementarity between the modalities. Considering that the importance of different loss components may vary over the training process and specific tasks, a dynamic weight balancing strategy was designed to automatically adjust the weight coefficients of each component based on the gradient size of each loss, training progress, and verification performance. For example, in the early stages of training, more emphasis may be placed on basic classification tasks, while as training progresses, the weight of element detection and feature consistency is gradually increased. This multi-task joint optimization training method enables the model to learn multiple related tasks simultaneously, which not only improves overall performance but also enhances the model's generalization and robustness.

[0070] In the joint training of the LPLC2-BERT fusion model, a carefully designed multi-task learning strategy and loss function are adopted. The joint loss function consists of three main components: classification loss, detection loss, and feature consistency loss, which are weighted and summed using a weight coefficient.

[0071] The classification loss uses the cross-entropy loss function, calculating the difference between the predicted class probability and the true class label. The detection loss consists of two parts: a bounding box regression loss and an element-wise classification loss. The bounding box regression loss uses a smoothed L1 loss to calculate the difference between the predicted bounding box and the true bounding box. The element-wise classification loss uses a focal loss to mitigate class imbalance, with the focal parameter set to 2.0. The feature consistency loss calculates the cosine similarity between BERT features and LPLC2 features to ensure semantic consistency between the two feature representations.

[0072] The weight coefficients adopt a dynamic adjustment strategy, with the initial values ​​set to 1.0 for classification loss, 0.5 for detection loss, and 0.1 for feature consistency loss. They are then dynamically adjusted as the training progresses: in the early stages of training, emphasis is placed on learning the basic classification tasks; in the middle stages of training, the weight of the detection tasks is gradually increased; and in the later stages of training, the weight of feature consistency is increased.

[0073] In addition, a gradient balancing mechanism is designed to monitor the gradient norm of each task loss. When the gradient of a task is significantly larger than that of other tasks (more than 3 times), the gradient of the task is automatically scaled to ensure the balance of learning among tasks.

[0074] We used the AdamW optimizer with weight decay, with a learning rate of 1e-5 for BERT and 1e-4 for LPLC2, and a weight decay of 0.01. We used a cosine annealing strategy for the learning rate schedule, with a 10% warm-up period.

[0075] Step S4.15: Implement model quantization and pruning to reduce inference time. A non-maximum suppression algorithm is developed to filter redundant detection results. A logical association graph between detected elements is constructed, resulting in a structured common text understanding representation and an optimized common text classification model. To improve the efficiency and practicality of the model in practical applications, this paper implements a series of inference optimization and post-processing techniques. First, model size and computational complexity are reduced through model quantization and pruning techniques, including weight quantization (converting 32-bit floating-point numbers to 8-bit integers), structural pruning (removing unimportant neuronal connections), and knowledge distillation (transferring knowledge from large models to smaller models). These techniques significantly reduce the model's storage requirements and inference time, enabling efficient operation in resource-constrained environments. Second, a non-maximum suppression (NMS) algorithm is developed for text element detection to filter redundant detection results. When multiple detection boxes overlap, this algorithm retains the one with the highest confidence and suppresses the remaining overlapping boxes, ensuring that each element is detected only once. Furthermore, a logical association diagram is constructed between detected elements. By analyzing the positional relationships, semantic associations, and contextual dependencies between elements, the logical structures between them, such as inclusion, order, and causality, are inferred. This structured public text understanding representation not only provides information about the location and type of elements but also reveals the underlying logical relationships between them, enabling a more comprehensive and in-depth understanding of public text content and providing a solid foundation for subsequent public text analysis and intelligent services.

[0076] In one embodiment of the present invention, in step S5, the newly added multi-source scientific and technological public text data to be classified is automatically classified based on the public text classification model to obtain classification results; and data mining technology is used to analyze the distribution characteristics of the public texts, and a visualization tool is used to generate an analysis result chart, including: Step S5.1: Automatically classify newly collected multi-source scientific and technological public text data based on the public text classification model to obtain classification results. After model training is complete, this sub-step applies the trained classification model to real-world scenarios to automatically classify newly collected scientific and technological public text data. First, the newly collected data undergoes the same preprocessing process as the training data, including text cleaning, formatting, and feature extraction, to ensure that the data input format remains consistent with that used during training. The processed data is then input into the integrated classification model, which performs predictions along three dimensions (tool, domain, and hierarchy) and outputs a probability distribution for each category. The final classification label is determined based on a preset probability threshold (typically 0.5 or an empirical value). For multiple categories with similar probabilities, multiple labels can be assigned simultaneously, reflecting the multi-category nature of public text. In particular, "difficult samples" with low predicted probabilities or similar probabilities for multiple categories are automatically marked as "requiring manual review" and submitted to professional review to ensure the reliability of the classification results. In addition, a confidence assessment mechanism for classification results has been implemented. A confidence score is calculated based on the distribution characteristics of the predicted probability, providing a reference for subsequent analysis and application. To address the dynamic changes in the content and expression of public text, a regular update mechanism for the classification model has been designed. Newly annotated data is collected at regular intervals (e.g., quarterly) to fine-tune the model and ensure it adapts to the latest public text characteristics and expressions. This regularly updated classification mechanism, combining automatic classification with manual review, can efficiently and accurately process large amounts of newly added scientific and technological public text data, providing fundamental support for subsequent data analysis and intelligent services.

[0077] Step S5.2: Analyze the Distribution Characteristics of Public Documents Using Data Mining Techniques. After completing the automatic classification of public documents, this substep utilizes advanced data mining techniques to conduct a multi-dimensional analysis of the classification results, uncovering the distribution characteristics and evolution patterns of public documents. First, a trend analysis is conducted in the temporal dimension. Using time series analysis, the changing trends in the number of various types of public documents are examined, identifying cyclical fluctuations, sudden increases, and long-term evolutionary trends. For example, it is possible to discover the cyclical pattern of the concentrated release of certain types of public documents (such as those supporting scientific and technological innovation funds) during specific time periods each year (such as the beginning of the year or after the fiscal budget), or to capture the rapid growth trend of public documents related to a particular scientific and technological field (such as artificial intelligence) in recent years. Second, a regional analysis is conducted in the spatial dimension, comparing the distribution and density of public documents across different regions (such as provinces and urban agglomerations). This generates regional public document heat maps, visually demonstrating the intensity and focus of public document support in each region. This analysis helps identify regional differences in public documents and the competitive landscape, providing a reference for enterprise site selection and regional innovation research. In addition, a correlation analysis of the content dimension was conducted. Through association rule mining and network analysis methods, the correlation patterns and co-occurrence relationships between different types of public texts were explored, a public text association network was constructed, and the supporting relationships and policy combination patterns between different public text tools were revealed. For example, it was possible to discover the high correlation between talent introduction policies and scientific research funding support policies, or to identify that public texts in certain fields often use a specific combination of support methods. These multi-dimensional data mining analyses not only reveal the distribution patterns and evolution trends of scientific and technological public texts, but also provide decision makers with valuable data insights, helping them understand the overall pattern and development dynamics of public text support, and providing data support for the formulation and adjustment of scientific and technological innovation policies.

[0078] Step S5.3: Generate analysis results charts using visualization tools. After completing the data mining analysis, this substep utilizes advanced data visualization techniques to transform complex analysis results into intuitive, easy-to-understand charts, enhancing the data's readability and decision-making value. A variety of visualization tools and technologies were employed, including web-based interactive visualization libraries (such as Echarts and D3.js) and professional data visualization platforms (such as Tableau and PowerBI), selecting the most appropriate visualization method based on the characteristics of the different analysis results. First, for temporal trend analysis, dynamic time series charts were generated, including line charts, area charts, and heat calendar charts, to visually demonstrate the changing trends and cyclical characteristics of the number of various types of public texts over time. These charts support multi-dimensional filtering and comparison, allowing users to simultaneously view the temporal evolution of public texts of different types or regions. Second, for spatial distribution analysis, interactive geographic information visualizations were created, including regional heat maps, bubble maps, and layered maps, to demonstrate the distribution density and typological characteristics of public texts across different geographic regions. These maps support multi-level drill-down analysis, allowing users to drill down from a national view to detailed views at the provincial, municipal, and even district and county levels. Furthermore, for public text content and relevance analysis, visualizations such as network diagrams, Sankey diagrams, and co-occurrence matrices have been designed to showcase the strength and flow of associations between different types of public text. The network diagram, in particular, uses visual encodings of nodes and edges (such as size, color, and thickness) to intuitively reflect the core nodes and key connections in the public text network. To enhance user experience and analytical flexibility, an interactive data dashboard has been developed to support multi-dimensional data filtering, comparison, and exploration. Users can customize their analysis by selecting specific criteria, such as timeframes, regions, and public text types. These visualizations are not only displayed internally but can also be exported to multiple formats (such as PNG, PDF, and interactive HTML) for easy use in reports and presentations. These diverse visualizations transform complex data analysis results into intuitive and clear visual information, significantly improving the data's comprehensibility and usability, and providing powerful visual support for scientific and technological innovation decision-making.

[0079] In one embodiment of the present invention, in step S6, a user profile is constructed based on user interest data including user registration information, browsing history, and search keyword data; a common text feature index is constructed based on the classification result and the enhanced text feature; and based on the user profile and the common text feature index, a recommendation algorithm is used to calculate the similarity between the user interest and the common text feature to generate personalized common text recommendation results, including: Step S6.1: Build a user profile based on user interest data, including user registration information, browsing history, and search keyword data. This substep is the foundation of personalized recommendation services and builds a comprehensive and accurate user interest profile through multi-source data fusion. First, user registration information is collected and analyzed, including basic attributes (such as industry, position, and professional background) and self-entered interest tags. This information provides an initial framework for user interests. Second, user browsing behavior data is continuously tracked and analyzed, including the types of public texts viewed, dwell time, favorites, and downloads. This implicit feedback reflects the user's actual interest tendencies and strength. Third, user search keyword data is recorded and analyzed. This explicit query information directly reflects the user's immediate information needs and focus. Based on this data collection, a user interest model is constructed using various machine learning techniques. First, a collaborative filtering algorithm is used to identify user groups with similar interests based on behavioral similarities, and the potential interests of individuals are inferred from group behavior. Second, content analysis techniques are applied to map the content features of the public texts that users have interacted with into an interest space, constructing a content-based interest representation. Third, topic models (such as LDA) are used to extract latent topics from user interaction text, forming a topic-level interest distribution. These different dimensional interest representations are integrated into a unified user profile, consisting of two components: long-term stable interests and short-term dynamic interests. Long-term interests reflect the user's basic professional focus and areas of ongoing attention, derived through the accumulation and smoothing of long-term behavioral data; short-term interests capture the user's immediate needs and attention shifts, primarily based on recent interactions and search history. A dynamic user profile update mechanism is also implemented, continuously adjusting the interest model based on the user's latest behavior, ensuring that the profile promptly reflects changes and developments in the user's interests. This multi-source data fusion and multi-dimensional modeling approach to user profile construction enables a comprehensive and accurate understanding of the user's interests and information needs, providing reliable user-side data support for subsequent personalized recommendations.

[0080] Step S6.2: Construct a public text feature index based on the classification results and the enhanced text features. While completing the user profile construction, this substep constructs an efficient feature index for public text data, providing data support for subsequent similarity matching and recommendation calculations. First, the classification results and enhanced text features from the previous steps are integrated to construct a multi-dimensional feature vector for each public text. These features include three-dimensional classification labels (tool dimension, domain dimension, and hierarchy dimension), text semantic vectors (document embeddings based on BERT or Word2Vec), and metadata features (such as publication time and issuing institution). To support efficient similarity calculation and retrieval, a hybrid index structure combining inverted indexes and vector indexes is adopted. For structured features such as classification labels and metadata, a traditional inverted index is established to support exact matching and range queries. For high-dimensional features such as text semantic vectors, an approximate nearest neighbor (ANN) search technique, such as the Hierarchical Navigable Small World (HNSW) or Annoy algorithm, is used to construct an efficient vector index, supporting fast similarity searches. This hybrid index structure ensures both the accuracy of queries and the efficiency of high-dimensional vector similarity calculations. In order to further improve the performance and flexibility of the index, a multi-level caching strategy and a dynamic update mechanism were implemented. The multi-level cache caches the features of popular public texts and common query results in memory, significantly reducing disk IO and computing overhead; the dynamic update mechanism ensures that newly added public texts can be indexed in a timely manner, and maintains the index structure through incremental updates, avoiding the high cost of full reconstruction. In addition, a feature weight adaptive adjustment mechanism is designed to dynamically adjust the weights of different features in similarity calculations based on feedback data from different user groups, so that the index can better adapt to the needs of different scenarios. This public text feature index with multi-dimensional feature fusion and efficient index structure provides strong content-side data support for subsequent personalized recommendations, ensuring the accuracy and efficiency of similarity calculations during the recommendation process.

[0081] Step S6.3: Based on the user profile and the public text feature index, a recommendation algorithm is used to calculate the similarity between the user's interests and the public text features, generating personalized public text recommendations. Once the user profile and public text feature index are in place, this sub-step implements an intelligent recommendation algorithm to generate personalized science and technology public text recommendations for the user. This hybrid recommendation approach combines multiple recommendation strategies to balance relevance, diversity, and novelty. First, content-based recommendation is the core strategy. This strategy calculates the similarity (e.g., cosine similarity, Euclidean distance, etc.) between the user's interest vector and the public text feature vector to identify the public text that best matches the user's interests. This calculation considers multiple dimensions of feature matching, including topic relevance, tool type matching, and applicability level conformance, dynamically adjusting the weights of each dimension based on historical user feedback. Second, collaborative filtering recommendation serves as a supplementary strategy. Based on the assumption that users with similar interests are likely to be interested in similar public texts, this strategy identifies potential recommendations based on positive feedback from users with similar interests. This approach can uncover implicit interest connections that are difficult to capture based on content, enriching the breadth of recommendation results. In addition, a knowledge graph-enhanced recommendation strategy was implemented. Leveraging a constructed public text knowledge graph, graph algorithms (such as random walks and path reasoning) were used to discover indirect connections between user interests and public text, further improving the accuracy and interpretability of recommendations. After generating the initial recommendation list, a multi-objective ranking optimization was performed. First, timeliness was considered, giving higher ranking weight to newly released, relevant public texts to ensure users have timely access to the latest policy information. Second, diversity was considered, using algorithms such as the Deterministic Point Process (DPP) to increase the diversity of recommendation results while ensuring relevance, avoiding overly homogeneous recommendations. Third, personalized factors were considered, adjusting the recommendation ranking based on user attributes (such as region and company size), prioritizing public texts that are more relevant to the user's specific situation. Finally, a recommendation explanation generation module was designed to automatically generate personalized reasons for each recommendation, such as "based on your interest in the field of artificial intelligence" or "related to the scientific research funding policy you recently reviewed," enhancing the transparency of the recommendations and user trust. Through this multi-strategy fusion and multi-objective optimization recommendation method, we can provide users with highly personalized, content-rich and timely updated science and technology public text recommendation services, effectively meeting the diverse needs of different users for public text information in the process of scientific and technological innovation.

[0082] To comprehensively evaluate the model's performance, detailed testing and comparisons were conducted across multiple dimensions. On common text classification tasks, the LPLC2-BERT fusion model significantly outperformed existing techniques. Compared to the traditional TF-IDF plus SVM approach, the model achieved a 10.6 percentage point improvement in average F1 (from 84.8% to 95.4%), and a 2.8 percentage point improvement compared to the base BERT model (from 92.6% to 95.4%). Across the tool, domain, and hierarchical dimensions, the model achieved F1 scores of 95.2%, 93.8%, and 97.1%, respectively, comprehensively outperforming existing methods.

[0083] The LPLC2-BERT model demonstrated excellent performance in the common text element detection task. For the four key elements of common text: objectives, implementation entities, supporting measures, and constraints, the model achieved an average precision of 90.7%, an average recall of 88.9%, an average F1 score of 89.8%, and an average detection accuracy (mAP@0.5) of 87.8%. Among them, implementation entity detection achieved the highest performance (F1 score of 93.3%), while constraint detection performance was relatively low (F1 score of 86.4%), but still maintained a high level.

[0084] In the evaluation of recommendation systems, the multi-objective optimization recommendation algorithm achieved significant improvements over traditional recommendation methods. Compared to content-based recommendations, the click-through rate increased by 39.6% (from 8.3% to 12.7%); compared to collaborative filtering, the precision increased by 15.7% and the recall increased by 14.0%. This method also performed particularly well in terms of diversity metrics, with coverage increasing by 39.8% and novelty by 42.4%. User satisfaction scores increased from 3.8 to 4.3 on a 5-point scale, a 13.2% improvement.

[0085] In addition, a case study was conducted to demonstrate the model's effectiveness in applying it to actual public documents. For example, in a public document regarding the "Management Measures for the Special Fund for the Innovation and Development of the Artificial Intelligence Industry," the model not only correctly classified it as "Funding Support Category / Artificial Intelligence Field / Provincial Level" but also pinpointed key elements within the document, including the document's objectives ("Promote the deep integration of AI with the real economy and accelerate the innovative application of AI" with a confidence level of 95.3%), implementing entities ("Provincial Department of Science and Technology, Provincial Department of Finance" with a confidence level of 97.1%), support measures ("Up to 5 million yuan in funding for AI core technology R&D projects" and "Up to 3 million yuan in funding for innovation platform construction" with confidence levels of 93.4% and 92.1%), and constraints ("Applicants must be corporate entities registered for at least two years" and "Projects must possess independent intellectual property rights" with confidence levels of 90.2% and 89.7%).

[0086] This precise element detection capability allows users to quickly locate the content of most concern in public texts, greatly improving the efficiency of information acquisition.

[0087] like Figure 2 As shown, the present invention also provides a deep learning-based intelligent classification and service device for scientific and technological public texts, including: The data collection and preprocessing module 701 is used to obtain multi-source scientific and technological public text data, and perform text cleaning, deduplication and format unification processing on the multi-source scientific and technological public text data to obtain a preprocessed multi-source scientific and technological public text data set; A feature extraction enhancement module 702 is configured to extract keywords based on the preprocessed multi-source science and technology public text dataset using a keyword extraction algorithm and combine the keywords with the public text titles to form enhanced text features; A classification system construction module 703 is used to construct a multi-dimensional public text classification system including talent support, financial support, technical support, and legal control based on the enhanced text features, perform data annotation, and establish a training data set; A model training module 704 is configured to perform feature extraction and domain adaptability fine-tuning on the training dataset using a BERT pre-trained model, and obtain a common text classification model through parameter optimization and ensemble learning; The intelligent analysis and visualization module 705 is used to automatically classify the newly added multi-source scientific and technological public text data to be classified based on the public text classification model to obtain classification results; and use data mining technology to analyze the distribution characteristics of public texts and generate analysis result charts through visualization tools; The recommendation service module 706 is used to build a user portrait based on user interest data including user registration information, browsing history and search keyword data; and to build a public text feature index based on the classification results and the enhanced text features; and based on the user portrait and the public text feature index, use a recommendation algorithm to calculate the similarity between user interests and public text features to generate personalized public text recommendation results.

[0088] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A deep learning-based intelligent classification and service method for scientific and technological public texts, characterized by: include: Acquire multi-source science and technology public text data, perform text cleaning, deduplication and format unification processing on the multi-source science and technology public text data, and obtain a pre-processed multi-source science and technology public text data set; Based on the preprocessed multi-source science and technology public text dataset, a keyword extraction algorithm is used to extract keywords and splice them with public text titles to form enhanced text features; Based on the enhanced text features, a multi-dimensional public text classification system including talent support, financial support, technical support, and regulatory control is constructed, and data annotation is performed to establish a training data set; The BERT pre-trained model is used to perform feature extraction and domain adaptability fine-tuning on the training dataset, and a common text classification model is obtained through parameter optimization and ensemble learning; Automatically classify the newly added multi-source scientific and technological public text data to be classified based on the public text classification model to obtain classification results; and use data mining technology to analyze the distribution characteristics of public texts, and generate analysis result charts through visualization tools; Based on user interest data including user registration information, browsing history and search keyword data, a user portrait is constructed; and based on the classification results and the enhanced text features, a public text feature index is constructed; and based on the user portrait and the public text feature index, a recommendation algorithm is used to calculate the similarity between user interests and public text features to generate personalized public text recommendation results.

2. The method according to claim 1, characterized in that The multi-source science and technology public text data is cleaned, deduplicated, and formatted uniformly to obtain a pre-processed multi-source science and technology public text data set, including: Utilize an automated data collection framework to collect data from websites at all levels, call API interfaces to obtain structured public text information, and form an original multi-source science and technology public text dataset containing title, document number, release date, and text content; The original multi-source science and technology public text dataset is subjected to HTML tag removal and format normalization processing, and duplicate public texts are removed by a hash algorithm to obtain the preprocessed multi-source science and technology public text dataset.

3. The method according to claim 1, characterized in that It also includes structural analysis of the preprocessed multi-source science and technology public text dataset: Based on the characteristics of public text formats, the preprocessed multi-source science and technology public text dataset is decomposed into title, text, and attachment parts, and key information including public text goals, specific measures, and implementation time are extracted from the text to form structured multi-source science and technology public text data; The structured multi-source scientific and technological public text data is subjected to terminology normalization and format unification processing to obtain a standardized multi-source scientific and technological public text data set.

4. The method according to claim 2, characterized in that The keyword extraction algorithm is used to extract keywords and combine them with the public text title to form enhanced text features, including: Obtain a science and technology public text domain dictionary, and use a Chinese word segmentation tool in combination with the science and technology public text domain dictionary to perform word segmentation processing on the preprocessed multi-source science and technology public text dataset to obtain a public text word segmentation result set; Extracting a keyword set representing the core content of the text from the public text word segmentation result set based on the TextRank algorithm; The keyword set is concatenated with the title of the preprocessed multi-source science and technology public text dataset, and converted into a vector representation using word embedding technology to obtain the enhanced text feature.

5. The method according to claim 1, wherein The construction of a multi-dimensional public text classification system including talent support, financial support, technical support, and regulatory control, as well as data annotation and the establishment of a training data set, includes: Design the multi-dimensional public text classification system including the public text tool dimension, the public text field dimension, and the public text level dimension, and formulate classification standards and judgment rules; A stratified sampling method is adopted to select public text samples, and domain experts are organized to manually annotate according to the classification standard and the judgment rule. The annotation quality is ensured through cross-annotation by multiple people to obtain the training data set.

6. The method according to claim 1, characterized in that The training dataset is subjected to feature extraction and domain adaptability fine-tuning using the BERT pre-training model, and a common text classification model is obtained through parameter optimization and ensemble learning, including: Dividing the training data set into a training set and a test set, performing BERT word segmentation processing on the text data of the training data set, and expanding the training samples through data augmentation operations; Fine-tune the Chinese BERT model for domain adaptability, setting parameters including learning rate and batch size, and using the cross-entropy loss function and Adam optimizer for model training; An ensemble learning method is used to combine the prediction results of multiple models to improve classification accuracy and obtain the common text classification model.

7. The method according to claim 1, characterized in that The BERT pre-trained model is used for feature extraction and domain adaptation fine-tuning, including: Based on the preprocessed multi-source science and technology public text dataset, a multi-dimensional feature space including semantic relevance, time relevance, and institutional relevance is constructed, and semantic vectors are calculated and the influence of public texts is evaluated using a word embedding method; An incremental learning strategy is adopted to enable the public text classification model to continuously learn new public text information, an anomaly detection algorithm is used to identify emerging concepts to dynamically expand the feature space, and a public text knowledge graph is established to achieve automatic expansion of concept boundaries; Multiple optimization objectives including classification accuracy, category balance, and diversity are defined, and a multi-objective optimization algorithm is used to find the balance point of each objective to obtain a public text classification model with multi-objective optimization.

8. The method according to claim 7, characterized in that The multi-objective quality diversity optimization further includes: A diversity balance strategy is introduced to ensure that the personalized public text recommendation results cover different types of public text information, and a diversity reward mechanism is designed to encourage the multi-objective optimized public text classification model to explore public text areas that users have not been exposed to; An attention mechanism is used to dynamically evaluate the importance weight of each feature in the multi-dimensional feature space, and a feature feedback loop is established to adjust the feature weight system through user interaction data; Expand traditional evaluation indicators and introduce diversity indicators including coverage and novelty to build a diversity-aware evaluation system.

9. The method according to claim 1, characterized in that The acquisition of multi-source scientific and technological public text data includes ensuring data collection stability through heterogeneous resource allocation, including: Develop a data collection engine that supports multiple protocols and is compatible with network protocols including HTTP, HTTPS, and FTP. Implement differentiated access control strategies to dynamically adjust collection parameters based on the characteristics of different data sources, build a distributed proxy pool, and obtain stable multi-source scientific and technological public text data collection capabilities. A microservices architecture is established based on container technology. By real-time monitoring and collecting resource usage, a priority task scheduling algorithm is designed to ensure that key data collection tasks are prioritized, thus achieving resource-aware dynamic scheduling. Build a dependency graph for data collection services, apply graph optimization algorithms to calculate the optimal service deployment method, establish edge nodes for data preprocessing, and obtain an optimized multi-source scientific and technological public text data collection architecture.

10. The method according to claim 9, characterized in that The heterogeneous resource allocation also includes quality of service assurance for data transmission and processing, including: The design includes multi-level service quality definitions, including high-priority real-time collection, standard collection, and background batch processing, enabling dynamic perception of network conditions. Adaptive data compression and transmission strategies are adopted according to different QoS levels to ensure the stability and integrity of data collection. Establish a terminal device feature database, develop an adaptive data format conversion engine, implement data processing optimization based on network conditions, and design a progressive data loading strategy to ensure data acquisition efficiency in different network environments; Deploy a distributed data collection and monitoring system, design a data collection performance monitoring panel, implement an intelligent alarm mechanism to automatically trigger the repair process when the data collection quality falls below the threshold, and build an end-to-end data collection quality assurance system.

11. The method according to claim 1, wherein The method further comprises: When using the BERT pre-trained model for feature extraction and domain adaptation fine-tuning, the LPLC2 neural network is integrated to achieve multi-target detection of public text content, including: Divide the preprocessed multi-source technology public text dataset into text blocks according to sentences and paragraphs, convert the text blocks into high-dimensional vectors using BERT word embedding technology, map them to two-dimensional space through a dimensionality reduction algorithm to retain semantic similarity, construct a two-dimensional semantic feature map, and realize two-dimensional representation conversion of public text; Define key element types including public text objectives, implementation entities, support measures, and constraints; design multi-element annotation specifications; develop auxiliary annotation tools to support the annotation of text areas; and establish a fine-grained public text multi-element annotation system; The BERT model is used to extract the contextual semantic representation of public text. The BERT hidden layer features are converted into two-dimensional feature maps as the input of the LPLC2 network. The local pyramid and contrast modules of LPLC2 are applied to enhance the recognition of key element boundaries. A multi-task learning head is designed to simultaneously output the overall classification and element detection results, and the LPLC2-BERT fusion public text classification model is obtained.

12. The method according to claim 11, characterized in that The LPLC2-BERT fusion public text classification model also includes attention mechanism enhancement and joint optimization, including: A self-attention mechanism is introduced in the BERT encoding layer to capture long-range dependencies between common text words. A cross-modal attention module is designed to enable the LPLC2 network to focus on key areas of the feature map based on semantic importance. This combines channel attention and spatial attention to enhance feature representation. A hierarchical attention fusion mechanism is constructed to integrate features at different levels to achieve attention-enhanced feature extraction. Construct a joint loss function that includes a common text classification loss and an element detection loss, introduce the consistency loss between BERT representation and LPLC2 features, and design a dynamic weight balancing strategy to adjust the contribution ratio of each loss to achieve multi-task joint optimization training; Implement model quantization and pruning to reduce inference time, develop a non-maximum suppression algorithm to filter redundant detection results, construct a logical association relationship diagram between detection elements, and obtain a structured public text understanding representation and an optimized public text classification model.

13. A deep learning-based intelligent classification and service device for scientific and technological public texts, characterized in that: include: A data collection and preprocessing module is used to obtain multi-source scientific and technological public text data, and perform text cleaning, deduplication and format unification processing on the multi-source scientific and technological public text data to obtain a preprocessed multi-source scientific and technological public text data set; A feature extraction enhancement module is used to extract keywords based on the preprocessed multi-source science and technology public text dataset using a keyword extraction algorithm and combine them with the public text titles to form enhanced text features; A classification system construction module is used to build a multi-dimensional public text classification system including talent support, financial support, technical support, and legal control based on the enhanced text features, and to perform data annotation and establish a training data set; A model training module is used to perform feature extraction and domain adaptability fine-tuning on the training dataset using the BERT pre-trained model, and obtain a common text classification model through parameter optimization and ensemble learning; An intelligent analysis and visualization module is used to automatically classify the newly added multi-source scientific and technological public text data to be classified based on the public text classification model to obtain classification results; and use data mining technology to analyze the distribution characteristics of public texts and generate analysis result charts through visualization tools; The recommendation service module is used to build a user portrait based on user interest data including user registration information, browsing history and search keyword data; and to build a public text feature index based on the classification results and the enhanced text features; and based on the user portrait and the public text feature index, use a recommendation algorithm to calculate the similarity between user interests and public text features to generate personalized public text recommendation results.

Images