Deep learning-based technology public text intelligent classification and service method and device

Abstract

The application discloses a kind of based on deep learning's science and technology public text intelligent classification and service method and device, comprising: obtaining multi-source science and technology public text data and pre-processing;Key word extraction algorithm is extracted key word and is spliced with public text title to form enhanced text feature;Multi-dimensional public text classification system is constructed and data labeling is carried out;Classifying model is obtained by using BERT pre-training model for feature extraction and fine-tuning;Newly added public text is automatically classified and visualized presentation;Based on user portrait and public text feature index, personalized recommendation result is generated.The application also relates to the technical solutions of multi-objective quality diversity optimization, heterogeneous resource allocation and LPLC2 neural network and BERT fusion.The application solves the technical problems that traditional methods have limited complex semantic understanding ability, single classification dimension and lack of integrated solutions, improves the accuracy of science and technology public text classification and the intelligent level of service.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing and deep learning, in particular to a deep learning-based intelligent classification and service method and device for public science and technology texts. BACKGROUND

[0002] Public science and technology text classification and intelligent service is an important field of public service department management and enterprise innovation. With the increasing number of public science and technology texts and the increasing complexity of their content, how to efficiently and accurately classify and provide intelligent services for public science and technology texts has become a technical problem to be solved.

[0003] The current common public science and technology text classification technology mainly uses keyword matching and traditional machine learning methods. For example, statistical methods such as TF-IDF are used to extract text features, combined with SVM, random forest, etc. to build a model, or a rule-based method is used for public science and technology text classification and service. These methods can achieve certain results in specific scenarios, but have limited ability to understand complex semantics.

[0004] With the development of natural language processing technology, pre-trained language models are introduced into the field of public science and technology text classification, which can capture the semantic features of public science and technology texts through deep neural network architecture. This method uses the context understanding ability of the language model to better grasp the semantic connotation of the public science and technology text, thereby improving the accuracy of classification. However, the existing technology has several obvious problems in the process of public science and technology text classification: first, the structure characteristics of public science and technology texts are not fully considered, making it difficult to accurately extract key information; second, the classification dimension is relatively single, which cannot meet the needs of multi-dimensional public science and technology analysis; third, there is a lack of integrated solutions from data processing to intelligent services. SUMMARY

[0005] The purpose of the present application is to provide a deep learning-based intelligent classification and service method and device for public science and technology texts, which aims to solve the technical problems of insufficient consideration of the structure characteristics of public science and technology texts, single classification dimension, and lack of integrated solutions in the prior art.

[0006] To achieve the above-mentioned purpose, the technical solutions provided by the present application are as follows:

[0007] A deep learning-based intelligent classification and service method for public science and technology texts, comprising:

[0008] Obtaining multi-source public science and technology text data, performing text cleaning, deduplication, and format unification processing on the multi-source public science and technology text data to obtain a preprocessed multi-source public science and technology text dataset;

[0009] Based on the pre-processed multi-source science and technology public text data set, a keyword extraction algorithm is used to extract keywords and splice them with public text titles to form enhanced text features;

[0010] Based on the enhanced text features, a multi-dimensional public text classification system including talent support, fund support, technology support and regulation control is constructed and data is labeled to establish a training data set;

[0011] The training data set is subjected to feature extraction and field adaptability fine-tuning by using a BERT pre-training model, and through parameter optimization and ensemble learning, a public text classification model is obtained;

[0012] Based on the public text classification model, the newly added multi-source science and technology public text data to be classified is automatically classified to obtain the classification result; and data mining technology is used to analyze the distribution characteristics of public text, and a visual tool is used to generate analysis result charts;

[0013] Based on the user interest data including user registration information, browsing history and search keyword data, a user portrait is constructed; and based on the classification result and the enhanced text features, a public text feature index is constructed; and based on the user portrait and the public text feature index, a recommendation algorithm is used to calculate the similarity between user interest and public text features to generate personalized public text recommendation results.

[0014] Preferably, the text cleaning, deduplication and format unification processing of the multi-source science and technology public text data to obtain the pre-processed multi-source science and technology public text data set comprises:

[0015] An automatic data acquisition framework is used to collect data from websites at all levels, API interfaces are called to obtain structured public text information, and an original multi-source science and technology public text data set containing titles, publication numbers, publication dates and content is formed;

[0016] The original multi-source science and technology public text data set is subjected to HTML tag removal and format standardization processing, and repeated public texts are removed by using a hash algorithm to obtain the pre-processed multi-source science and technology public text data set.

[0017] Preferably, it further comprises structured parsing of the pre-processed multi-source science and technology public text data set:

[0018] Based on the format characteristics of public texts, the pre-processed multi-source science and technology public text data set is decomposed into titles, texts and attachment parts, and key information including public text targets, specific measures and implementation times is extracted from the texts to form a structured multi-source science and technology public text data;

[0019] The structured multi-source public text data of science and technology is subjected to term standardization and format unification processing to obtain a standardized multi-source public text data set of science and technology.

[0020] Preferably, the keyword extraction algorithm is used to extract keywords and splice them with public text titles to form enhanced text features, including:

[0021] A science and technology public text field dictionary is obtained, and a Chinese word segmentation tool is combined with the science and technology public text field dictionary to perform word segmentation processing on the preprocessed multi-source public text data set of science and technology to obtain a public text word segmentation result set.

[0022] A keyword set representing the core content of the text is extracted from the public text word segmentation result set based on a TextRank algorithm.

[0023] The keyword set is spliced with the title of the preprocessed multi-source public text data set of science and technology, and word embedding technology is used to convert it into a vector representation to obtain the enhanced text features.

[0024] Preferably, a multi-dimensional public text classification system including talent support, financial support, technical support, and regulatory control is constructed, and data labeling is performed to establish a training data set, including:

[0025] The multi-dimensional public text classification system including a public text tool dimension, a public text field dimension, and a public text level dimension is designed, and classification standards and judgment rules are formulated.

[0026] A stratified sampling method is used to select public text samples, and field experts manually label them according to the classification standards and judgment rules. Multiple cross-labeling is used to ensure labeling quality to obtain the training data set.

[0027] Preferably, the training data set is subjected to feature extraction and field adaptability fine-tuning using a BERT pre-training model, and a public text classification model is obtained through parameter optimization and ensemble learning, including:

[0028] The training data set is divided into a training set and a test set, and the text data of the training data set is subjected to BERT tokenizer processing, and the training samples are expanded through data augmentation operations.

[0029] The field adaptability fine-tuning is performed based on a Chinese BERT model, parameters including learning rate and batch size are set, and a cross-entropy loss function and an Adam optimizer are used for model training.

[0030] An ensemble learning method is used to combine the prediction results of multiple models to improve classification accuracy to obtain the public text classification model.

[0031] Preferably, the feature extraction and domain adaptation fine-tuning using the BERT pre-training model comprises:

[0032] Based on the pre-processed multi-source scientific public text data set, a multi-dimensional feature space containing semantic correlation, time correlation and institution correlation is constructed, semantic vectors are calculated by word embedding method and public text influence is evaluated;

[0033] An incremental learning strategy is used to make the public text classification model continuously learn new public text information, an anomaly detection algorithm is used to identify emerging concepts to dynamically expand the feature space, and a public text knowledge graph is established to realize automatic expansion of concept boundaries;

[0034] A plurality of optimization objectives including classification accuracy, class balance and diversity are defined, a multi-objective optimization algorithm is used to find the balance point of each objective, and a multi-objective optimized public text classification model is obtained.

[0035] Preferably, the multi-objective quality diversity optimization further comprises:

[0036] A diversity balance strategy is introduced to ensure that the personalized public text recommendation result covers different types of public text information, and a diversity reward mechanism is designed to encourage the multi-objective optimized public text classification model to explore public text fields that the user has not contacted;

[0037] An attention mechanism is used to dynamically evaluate the importance weight of each feature of the multi-dimensional feature space, and a feature feedback loop is established to adjust the feature weight system through user interaction data;

[0038] Traditional evaluation indicators are expanded to introduce diversity indicators including coverage and novelty, and a diversity-aware evaluation system is constructed.

[0039] Preferably, the acquisition of multi-source scientific public text data comprises ensuring data collection stability through heterogeneous resource allocation, comprising:

[0040] A multi-protocol supported data collection engine is developed to support network protocols including HTTP, HTTPS and FTP, a differentiated access control strategy is implemented to dynamically adjust collection parameters according to different data source characteristics, a distributed proxy pool is constructed, and stable multi-source scientific public text data collection capability is obtained;

[0041] A micro-service architecture is established based on container technology, resource usage is collected through real-time monitoring, a priority task scheduling algorithm is designed to ensure that key data collection tasks are executed first, and resource-aware dynamic scheduling is realized;

[0042] A dependency graph of the data collection service is constructed, a graph optimization algorithm is applied to calculate the optimal service deployment mode, an edge node is established for data preprocessing, and an optimized multi-source scientific public text data collection architecture is obtained.

[0043] Preferably, the heterogeneous resource allocation further includes quality of service assurance for data transmission and processing, including:

[0044] The design includes multi-level service quality definitions, including high-priority real-time collection, standard collection, and background batch processing, enabling dynamic perception of network conditions. Adaptive data compression and transmission strategies are adopted according to different QoS levels to ensure the stability and integrity of data collection.

[0045] Establish a terminal device feature database, develop an adaptive data format conversion engine, implement data processing optimization based on network conditions, and design a progressive data loading strategy to ensure data acquisition efficiency in different network environments;

[0046] Deploy a distributed data collection and monitoring system, design a data collection performance monitoring panel, implement an intelligent alarm mechanism to automatically trigger the repair process when the data collection quality falls below the threshold, and build an end-to-end data collection quality assurance system.

[0047] Preferably, the method further comprises:

[0048] The BERT pre-trained model is used for feature extraction and domain adaptation fine-tuning, and the LPLC2 neural network is integrated to achieve multi-target detection of public text content, including:

[0049] Divide the preprocessed multi-source technology public text dataset into text blocks according to sentences and paragraphs, convert the text blocks into high-dimensional vectors using BERT word embedding technology, map them to two-dimensional space through a dimensionality reduction algorithm to retain semantic similarity, construct a two-dimensional semantic feature map, and realize two-dimensional representation conversion of public text;

[0050] Define key element types including public text objectives, implementation entities, support measures, and constraints; design multi-element annotation specifications; develop auxiliary annotation tools to support the annotation of text areas; and establish a fine-grained public text multi-element annotation system;

[0051] The BERT model is used to extract the contextual semantic representation of public text. The BERT hidden layer features are converted into two-dimensional feature maps as the input of the LPLC2 network. The local pyramid and contrast modules of LPLC2 are applied to enhance the recognition of key element boundaries. A multi-task learning head is designed to simultaneously output the overall classification and element detection results, and the LPLC2-BERT fusion public text classification model is obtained.

[0052] Preferably, the LPLC2-BERT fused public text classification model further includes attention mechanism enhancement and joint optimization, including:

[0053] The self-attention mechanism is introduced into the BERT coding layer to capture the long-distance dependency relationship between the public text words, the cross-modal attention module is designed to enable the LPLC2 network to focus on the key area of the feature map according to the semantic importance, the channel attention and the spatial attention are combined to enhance the feature representation, the hierarchical attention fusion mechanism is constructed to integrate different levels of features, and the attention-enhanced feature extraction is realized;

[0054] A joint loss function including a public text classification loss and an element detection loss is constructed, a consistency loss between the BERT representation and the LPLC2 feature is introduced, a dynamic weight balancing strategy is designed to adjust the contribution proportion of each loss, and multi-task joint optimization training is realized;

[0055] Model quantization and pruning are realized to reduce the inference time, a non-maximum suppression algorithm is developed to filter redundant detection results, a logical association relationship graph between the detected elements is constructed, and a structured public text understanding representation and an optimized public text classification model are obtained.

[0056] The application also provides a deep learning-based scientific and technological public text intelligent classification and service device, which comprises:

[0057] A data acquisition and preprocessing module is used to acquire multi-source scientific and technological public text data, perform text cleaning, deduplication and format unification processing on the multi-source scientific and technological public text data, and obtain a preprocessed multi-source scientific and technological public text data set.

[0058] A feature extraction and enhancement module is used to extract keywords based on the preprocessed multi-source scientific and technological public text data set, and the keywords are spliced with public text titles to form enhanced text features.

[0059] A classification system construction module is used to construct a multi-dimensional public text classification system including talent support, fund support, technical support and regulation control based on the enhanced text features, and perform data labeling to establish a training data set.

[0060] A model training module is used to extract features and adapt to the field by using a BERT pre-training model on the training data set, and obtain a public text classification model through parameter optimization and ensemble learning.

[0061] An intelligent analysis and visualization module is used to automatically classify newly added multi-source scientific and technological public text data to be classified based on the public text classification model, and obtain a classification result; and data mining technology is used to analyze the distribution characteristics of public texts, and a visualization tool is used to generate an analysis result chart.

[0062] The recommendation service module is configured to construct a user portrait based on user interest data including user registration information, browsing history and search keyword data, construct a public text feature index based on the classification result and the enhanced text feature, and calculate the similarity between user interest and public text features based on the user portrait and the public text feature index to generate a personalized public text recommendation result.

[0063] The present application has the following advantages:

[0064] 1. The present application designs an automatic data collection program and API calling method for different levels of public service department websites through multi-source public text data collection and structured parsing technology, and extracts the key components of public text through a structured parsing algorithm to realize standardized processing of public text data.

[0065] 2. The public text feature enhancement method proposed in the present application effectively improves the understanding ability of the subsequent classification model for public text by splicing the public text title and keywords to form an enhanced text feature through domain dictionary construction and specific keyword extraction algorithm.

[0066] 3. The present application innovatively designs a science and technology public text classification system including multiple dimensions such as public text tools, public text fields, public text levels, and formulates detailed classification standards and judgment rules, laying a foundation for accurate classification of public text.

[0067] 4. The present application uses BERT pre-training model for domain adaptability fine-tuning according to the characteristics of science and technology public text, and improves the classification accuracy through ensemble learning method, solving the problem of public text semantic understanding.

[0068] 5. The present application constructs a public text association network based on the citation relationship and theme similarity of public text, identifies core public text nodes and public text clusters through social network analysis method, and reveals the complex relationship and evolution path between public texts.

[0069] 6. The present application designs a public text recommendation algorithm considering timeliness and relevance by combining user portrait and public text feature representation, realizes accurate science and technology public text information push service, and meets the personalized needs of different users. BRIEF DESCRIPTION OF DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0071] Figure 1 is a flow chart of the method for intelligent classification and service of public scientific and technological texts based on deep learning of the present application;

[0072] Figure 2 is a structural diagram of the device for intelligent classification and service of public scientific and technological texts based on deep learning of the present application. DETAILED DESCRIPTION

[0073] The present application will be further described in detail below in combination with the drawings and examples.

[0074] As shown in Figure 1 , the method for intelligent classification and service of public scientific and technological texts based on deep learning provided by the present application comprises the following steps:

[0075] Step S1: Obtain multi-source public scientific and technological text data, and perform text cleaning, deduplication and format unification processing on the multi-source public scientific and technological text data to obtain a preprocessed multi-source public scientific and technological text data set. This step is the basic link of the entire method for intelligent classification and service of public scientific and technological texts. First, scientific and technological text data is obtained through various channels, including official websites of public service departments at all levels, professional databases and open API interfaces, etc. The original data obtained usually contains a large amount of noise and redundant information, so it needs to be processed in a progressive manner. Interfering elements such as HTML tags and special symbols are removed, and the text format is standardized. At the same time, a fingerprint of the text content is calculated using a hash algorithm to identify and remove duplicate public texts, ensuring the uniqueness and quality of the data set. In addition, the data from different sources needs to be unified in format, including character encoding unification, time format standardization and text structure standardization, etc., to finally form a preprocessed scientific and technological text data set with clear structure and unified format, providing a high-quality data basis for subsequent feature extraction and classification model training.

[0076] Step S2: Based on the pre-processed multi-source science and technology public text dataset, a keyword extraction algorithm is used to extract keywords and splice them with the public text title to form enhanced text features. After data preprocessing, this step aims to extract key features that can represent the core content of the public text. First, use a specially constructed science and technology public text field dictionary and Chinese word segmentation tool to accurately segment the pre-processed text and identify professional terms and important concepts. Then, apply algorithms such as TextRank to extract the keyword set that best represents the core content of the text from the segmentation results. These keywords usually summarize the theme and key points of the text. Next, splice the extracted keyword set with the title of the public text to form an enhanced text representation. This splicing strategy can retain both the summary information in the title and the detailed content in the body, making the text features more comprehensive and rich. Finally, use word embedding techniques such as Word2Vec to convert these text features into vector representations, and combine them with public text metadata features (such as publication time, publishing agency, etc.) to build a multi-dimensional feature representation, providing more rich input information for subsequent classification models and improving the model's understanding of public text semantics.

[0077] Step S3: Based on the enhanced text features, a multi-dimensional public text classification system including talent support, funding support, technology support, and regulation control is constructed and data is labeled to establish a training dataset. This step is the core of science and technology public text classification. First, according to the content characteristics of science and technology public text and actual application requirements, a multi-dimensional classification system is designed, including public text tool dimension (such as talent support, funding support, technology support, and regulation control), public text field dimension (such as artificial intelligence, biomedicine, and new materials), and public text level dimension (such as national, provincial, and municipal). This multi-dimensional classification system can comprehensively describe the characteristics and uses of science and technology public text from different angles. After the classification system is established, detailed classification standards and judgment rules are developed to clearly define the definition, scope, and judgment boundary of each category, ensuring the consistency and accuracy of the labeling process. Then, representative public text samples are selected from the pre-processed dataset using stratified sampling methods, and field experts are organized to manually label them according to the established standards. To ensure labeling quality, multiple cross-labeling methods are used to evaluate labeling quality by calculating labeling consistency indicators, and controversial samples are discussed and corrected. Through this series of strict labeling processes, a high-quality training dataset is finally established, providing reliable supervision signals for subsequent model training.

[0078] Step S4: Feature extraction and domain adaptation fine-tuning using the BERT pre-training model on the training data set, parameter optimization and ensemble learning to obtain a public text classification model. This step uses deep learning technology to build a public science and technology text classification model. First, the labeled training data set is divided into training set and test set according to a certain proportion, and the text data is preprocessed using the BERT tokenizer to convert it into the input format required by the model. At the same time, the training sample is expanded by data augmentation techniques such as synonym replacement, back translation, etc. to improve the generalization ability of the model. Then, based on the pre-trained Chinese BERT model, the model parameters are adjusted for domain adaptation fine-tuning, including the optimization of learning rate, batch size, training rounds, etc. In the training process, the cross-entropy loss function is used to evaluate the difference between the model prediction and the true label, and the Adam optimizer is used to update the model parameters, while learning rate decay and other strategies are applied to avoid overfitting. After training, the model performance is evaluated using the test set, and the accuracy, precision, recall, etc. are calculated, and the model is optimized according to the evaluation results. Finally, ensemble learning methods such as voting, stacking, etc. are used to combine the prediction results of multiple base models to further improve the classification accuracy, and an excellent public text classification model is obtained, which is deployed as a Web service to provide API interfaces for other calls.

[0079] Step S5: Based on the public text classification model, automatically classify the newly added multi-source science and technology public text data to obtain the classification results; and use data mining techniques to analyze the distribution characteristics of public text, and generate analysis result charts using visualization tools. After the model is trained, this step applies the trained classification model to actual scenarios to automatically classify and process newly added science and technology public text data. First, the new data is preprocessed and feature extracted in the same way as the training data, then the processed data is input into the classification model to obtain the classification prediction results and their confidence. For samples with low confidence, they will be marked for manual review to ensure the reliability of the classification results. After classification, data mining techniques are applied to analyze the classification results in multiple dimensions, including time dimension analysis (such as the publishing trend and periodic characteristics of each type of public text), regional dimension analysis (such as the distribution differences of public text in different regions), and public text type dimension analysis (such as the proportion and changes of each type of support policy). Through these analyses, hot areas and vacuum areas of public text can be identified to provide data support for decision-makers. Finally, use Echarts, D3.js, etc. visualization tools to present the analysis results in the form of intuitive charts, including public text distribution heat map, time evolution trend chart, correlation network diagram, etc. and design interactive data panels to support users to perform multi-dimensional data filtering and in-depth analysis, and generate comprehensive public text analysis visualization reports.

[0080] Step S6: based on the user interest data including user registration information, browsing history and search keyword data, construct a user portrait; and based on the classification result and the enhanced text feature, construct a public text feature index; and based on the user portrait and the public text feature index, use a recommendation algorithm to calculate the similarity between user interest and public text feature, and generate a personalized public text recommendation result. This step realizes the intelligent recommendation service of scientific and technological public text. First, the multi-dimensional user interest portrait is constructed by analyzing the user's registration information (such as industry, position, research field, etc.), browsing history (such as the type and content of the public text viewed) and search keyword data (reflecting the user's immediate interest point). These portraits can fully reflect the user's preferences and needs in the field of scientific and technological public text. At the same time, based on the classification result and the enhanced text feature in the foregoing steps, a feature vector is constructed for each public text, and an efficient feature index is established to support fast retrieval and matching. After the user portrait and the public text feature index are ready, the similarity between the user interest feature and the public text feature is calculated by using collaborative filtering, content-based recommendation and other algorithms to generate a preliminary recommendation list. Then, considering the timeliness (preferentially recommending newly released related public texts), relevance (matching degree with the user's current attention field) and other factors, the recommendation results are sorted and optimized to finally generate a personalized public text recommendation result. This intelligent recommendation service can help users quickly find the most relevant content to their needs among the vast amount of scientific and technological public text, greatly improving the efficiency and accuracy of information acquisition, and providing valuable decision support for researchers, enterprise innovators and policy makers.

[0081] In an embodiment of the present application, the multi-source scientific and technological public text data in step S1 is subjected to text cleaning, deduplication and format unification processing to obtain a pre-processed multi-source scientific and technological public text data set, which includes:

[0082] Step S1.1: Data collection from various levels of websites using an automated data collection framework, calling API interfaces to obtain structured public text information, forming an original multi-source science and technology public text dataset containing title, author, publication date, and content. This sub-step is the core work of the data collection phase. First, an automated data collection framework based on Python is developed, integrating tools such as Scrapy and Requests. Specialized crawler programs are designed for official websites of national, provincial, and municipal public service departments. These crawler programs can automatically identify website structures, locate public text publishing pages, and extract key information according to pre-set rules. To deal with structural differences and anti-crawler mechanisms of different websites, adaptive page parsing strategies and dynamic request parameter adjustments are implemented to ensure the stability and comprehensiveness of data collection. In addition to web crawling, open API interfaces provided by public service departments are called to directly obtain structured public text data. These APIs usually provide more standardized and complete data formats, which can supplement the information that may be missed by web crawling. Through the combination of these two channels, an original multi-source science and technology public text dataset containing key fields such as public text title, author, publication date, publishing agency, and content is formed. This dataset covers various types of public texts published by public service departments at all levels, such as science and technology policies, funding support plans, and talent introduction projects, laying a solid foundation for subsequent data processing and analysis.

[0083] Step S1.2: HTML tag removal and format normalization processing is performed on the original multi-source scientific public text data set, and repeated public texts are removed by a hash algorithm to obtain the pre-processed multi-source scientific public text data set. After obtaining the original data, this sub-step performs data cleaning and preprocessing work. First, for the text content crawled from the webpage, non-text elements such as HTML tags, JavaScript code and CSS styles are removed using regular expressions and tools such as BeautifulSoup, and pure text content is extracted. Then, the text is subjected to format normalization processing, including unified character encoding (converted to UTF-8), normalized white space (such as replacing extra spaces, line breaks), correcting special symbols (such as full-width half-width conversion, Chinese and English punctuation uniformity), etc. At the same time, regular expressions are used to identify and repair common format errors, such as inconsistent date formats, chaotic document number formats, etc. In order to ensure the uniqueness and quality of the data set, a hash algorithm based on SHA-256 is used to calculate a unique hash value for the core content (mainly the title and the main text) of each public text, which is used as the "fingerprint" of the text. By comparing these hash values, repeated public texts, including complete duplicates and highly similar content, can be efficiently identified and removed. This de-duplication process is particularly important because the same public text may appear in multiple websites or multiple versions, and de-duplication can significantly improve the efficiency and accuracy of subsequent processing. After these cleaning and normalization processes, a pre-processed multi-source scientific public text data set with uniform format and unique content is obtained, providing high-quality data input for subsequent feature extraction and model training.

[0084] In an embodiment of the present application, step S1 further comprises structuring analysis on the pre-processed multi-source scientific public text data set:

[0085] Step S1.3: Based on the characteristics of the public text format, the pre-processed multi-source science and technology public text data set is divided into title, body, and attachment parts, and key information including public text targets, specific measures, and implementation time is extracted from the body to form structured multi-source science and technology public text data. After basic text cleaning, this sub-step further structures the public text to mine its internal semantic structure and key information. First, according to the format characteristics of science and technology public text, a special structured parsing algorithm is designed, which can accurately identify and separate different components of public text, including title, body, and attachment list. For the title part, the theme and type information of the public text are extracted; for the body part, more in-depth semantic analysis is performed to identify paragraph structure and chapter division. In particular, for the body content, an information extraction module based on the combination of rules and machine learning is developed, which can automatically identify and extract key information elements in the public text, including the main target of the public text (such as "promote scientific and technological innovation", "support industrial development", etc.), specific support measures (such as amount of funds, support methods, application conditions, etc.), implementation time (such as policy validity period, application deadline, etc.), and responsible department, etc. These extracted information is organized according to the pre-defined structured mode to form structured public text data with clear structure and explicit semantics. This structured processing not only makes the content of the public text more organized and easy to understand, but also provides a more accurate information basis for subsequent feature extraction, classification analysis and intelligent recommendation, greatly improving the understanding depth and application value of the public text content;

[0086] Step S1.4: Term normalization and format unification processing on the structured multi-source scientific and technological public text data, obtaining a standardized multi-source scientific and technological public text data set. After completing the structured analysis of public text, this sub-step further standardizes the extracted structured data to ensure data consistency and comparability. First, for term normalization, a standard term library for the scientific and technological public text field is constructed, including common policy terms, scientific and technological terms, and professional expressions, and a synonym mapping table is established to unify different expressions of the same concept to the standard term. For example, different expressions such as "artificial intelligence", "AI", "intelligent technology" are unified to the standard term "artificial intelligence", ensuring consistency in subsequent analysis. At the same time, automatic conversion between traditional and simplified Chinese is realized, and all texts are unified into simplified Chinese format. In terms of format unification, time information is standardized, and various different formats of date representation (such as "2023-01-01", "2023.1.1", "2023 / 01 / 01", etc.) are converted to a unified standard format (such as "2023-01-01"), facilitating subsequent time series analysis. For document number format, similar standardization is also carried out, and different formats of document number (such as "Guofa 〔2023〕1", "Guofa

[2023] 1", etc.) are unified to the standard format. In addition, numerical representation is also unified, such as "1 million", "1,000 million", "1000 million" and other different representations are unified to the standard numerical format. Through this series of term normalization and format unification processing, a standardized scientific and technological public text data set with consistent format and standardized terms is finally obtained, which greatly improves the accuracy and efficiency of subsequent data analysis, providing a high-quality data foundation for model training and feature extraction.

[0087] In an embodiment of the present application, acquiring multi-source scientific and technological public text data includes ensuring data collection stability through heterogeneous resource allocation, including:

[0088] Step S1.5: Develop a multi-protocol supported data collection engine compatible with network protocols including HTTP, HTTPS, and FTP, implement differentiated access control policies to dynamically adjust collection parameters according to different data source characteristics, build a distributed proxy pool, and obtain stable multi-source scientific public text data collection capability. Specifically, for different public service department websites and data sources, the present application develops a set of data collection engine that can support multiple network protocols simultaneously. This engine not only supports standard HTTP and HTTPS protocols, but also supports FTP and other file transfer protocols, ensuring that scientific public text data can be obtained from various data sources. At the same time, in view of the characteristics of different data sources, such as access frequency limit, response time characteristics, etc., differentiated access control policies are implemented, which can dynamically adjust access frequency, concurrency and timeout settings, etc. according to the real-time monitoring of data source response. In addition, in order to avoid the interruption of data collection caused by IP being banned, a distributed proxy IP pool is built, and the data request is performed through intelligent rotation of proxy IP, effectively improving the stability and continuity of data collection, and ensuring the comprehensive acquisition of scientific public text data.

[0089] Step S1.6: Establish a micro-service architecture based on container technology, collect resource usage through real-time monitoring, design a priority task scheduling algorithm to ensure that critical data collection tasks are executed first, and realize resource-aware dynamic scheduling. In this embodiment, container technologies such as Docker are used to split data collection into multiple independent microservices, such as web crawling services, API calling services, and data cleaning services. Each service can be independently deployed and expanded. Through monitoring tools such as Prometheus, the CPU usage, memory occupation, network throughput and other resource usage of each microservice are collected in real time, and based on these monitoring data, a priority task scheduling algorithm is designed. This algorithm can allocate computing resources according to the importance and urgency of tasks, ensuring that critical data collection tasks (such as real-time updating of important public texts) can obtain resources and execute first, while non-critical tasks are reasonably delayed or resource-limited. Through this resource-aware dynamic scheduling mechanism, important data collection tasks can be completed smoothly in the case of limited resources, and the overall resource utilization efficiency is improved.

[0090] Step S1.7: Construct a dependency graph of the data collection service, apply a graph optimization algorithm to calculate the optimal service deployment mode, establish an edge node for data preprocessing, and obtain an optimized multi-source scientific public text data collection architecture. The present application constructs a complete service dependency graph by analyzing the calling relationship and data flow between each data collection service. This graph clearly shows the entire data flow process from the data source to the final storage. Based on this dependency graph, a graph optimization algorithm (such as the minimum spanning tree algorithm and network flow algorithm) is applied to calculate the optimal service deployment scheme, reducing unnecessary network transmission and service hopping. At the same time, edge computing nodes are deployed near the data source, which can perform preliminary data cleaning, format conversion and deduplication operations at the first time of data collection, greatly reducing the amount of data that needs to be transmitted to the center node. Through this optimized multi-source scientific public text data collection architecture, the efficiency and scalability of data collection are realized, which can cope with the growing amount of scientific public text data and complex data source environment.

[0091] In an embodiment of the present application, the heterogeneous resource allocation also includes quality of service guarantee for data transmission and processing, including:

[0092] Step S1.8: Design a multi-level service quality definition including high-priority real-time collection, standard collection and background batch processing, realize dynamic network status perception, adopt adaptive data compression and transmission strategies according to different QoS levels, and ensure the stability and integrity of data collection. According to the importance and timeliness requirements of scientific public text data, the present application designs a three-level quality of service (QoS) definition: high-priority real-time collection for important and time-sensitive public texts such as newly released national science and technology policies; standard collection for regular scientific public texts; background batch processing for supplementary collection and update of historical public texts. Through network detection technology, the current network status is monitored in real time, including bandwidth, delay and packet loss rate, etc. According to these indicators and the QoS level of the task, the most suitable data compression algorithm and transmission strategy are dynamically selected. For example, when the network condition is good, high-priority tasks can use algorithms with lower compression ratio to ensure data quality; while in network congestion, algorithms with higher compression ratio can be used to ensure timely data transmission. In addition, breakpoint resume and data integrity checking mechanisms are also implemented, so that even in the case of network interruption, the integrity of data collection can be guaranteed, and data loss or duplication will not occur.

[0093] Step S1.9: Establish a terminal device characteristic database, develop an adaptive data format conversion engine, implement data processing optimization based on network conditions, design a progressive data loading strategy, and ensure data acquisition efficiency in different network environments. To adapt to the characteristics of different terminal devices (such as servers, personal computers, mobile devices, etc.) and network environments, the present application establishes a characteristic database containing information such as the processing capacity, screen size, and network characteristics of various terminal devices. Based on this database, an adaptive data format conversion engine is developed, which can automatically adjust the format and size of the data according to the characteristics of the terminal device. For example, for mobile devices with limited processing capacity, a more lightweight data format is generated; while for high-performance servers, more detailed and complete data can be provided. At the same time, data processing optimization based on network conditions is implemented, and in the case of limited network bandwidth, critical data fields are prioritized for transmission, and non-critical information transmission is delayed. In addition, a progressive data loading strategy is designed, allowing users to view the loaded part before the data is completely loaded, greatly improving user experience and data acquisition efficiency. These measures together ensure efficient acquisition and processing of scientific public text data in various network environments.

[0094] Step S1.10: Deploy a distributed data collection monitoring system, design a data collection performance monitoring panel, implement an intelligent alarm mechanism to automatically trigger repair processes when data collection quality falls below a threshold, and build an end-to-end data collection quality assurance system. To comprehensively monitor the quality and performance of the data collection process, the present application deploys a distributed monitoring system that can track the execution status, time consumption, resource consumption, and data quality of each data collection task in real time. Based on these monitoring data, an intuitive performance monitoring panel is designed, which allows administrators to quickly understand the overall system operation and detailed information of each sub-task. At the same time, an intelligent alarm mechanism is implemented, which automatically triggers the corresponding repair process when the collection success rate, data integrity, or timeliness of a certain data source falls below the preset threshold, such as adjusting the collection parameters, switching the proxy IP, or notifying the administrator for manual intervention. In addition, complete log recording and analysis functions are established, which can discover potential problem patterns and optimization opportunities by mining historical data. These measures together form an end-to-end data collection quality assurance system, ensuring that the collection process of scientific public text data always maintains high quality and high reliability.

[0095] In one embodiment of the present application, the keyword extraction algorithm is used in step S2 to extract keywords and splice them with the public text title to form enhanced text features, including:

[0096] Step S2.1: Obtain a public science and technology text field dictionary, and use a Chinese word segmentation tool combined with the public science and technology text field dictionary to perform word segmentation processing on the preprocessed multi-source public science and technology text data set to obtain a public text word segmentation result set. This sub-step is the basic link of the feature extraction process. First, a special public science and technology text field dictionary is constructed through multiple channels. The construction of this dictionary integrates multiple sources: on the one hand, professional terms and standard expressions in the fields of science and technology policy, funding support, talent plan, etc. are collected and sorted, including standard expressions extracted from official documents; on the other hand, through statistical analysis of existing public text corpus, high-frequency professional terms and unique expressions are automatically identified using TF-IDF, PMI and other methods, and the dictionary content is constantly enriched and updated. The final field dictionary contains tens of thousands of professional terms, policy terms and specific expressions in the field of public science and technology text, providing important support for accurate word segmentation. In the word segmentation processing stage, mainstream Chinese word segmentation tools such as jieba are used, and the constructed field dictionary is imported as a custom dictionary, significantly improving the recognition accuracy of professional terms and policy expressions. In the word segmentation process, stop word filtering (removing common function words, conjunctions and other words that do not contribute much to the theme of the text) and part-of-speech tagging (identifying nouns, verbs, adjectives and other different parts of speech to provide more information for subsequent feature extraction) are also performed. In addition, synonym processing function is also realized, which can identify and merge similar professional terms, such as "subsidy", "funding support", "financial support" and other synonymous expressions, reducing the redundancy of feature space. Through these detailed word segmentation processing, a high-quality public text word segmentation result set is finally obtained, which accurately reflects the semantic content and professional term distribution of the public text, laying a solid foundation for subsequent keyword extraction and feature construction.

[0097] Step S2.2: Extracting keyword set representing core content of public text based on TextRank algorithm. After obtaining the segmentation results, this sub-step aims to extract the keywords that best represent the core content of each public text. The TextRank algorithm based on graph model is mainly adopted, which is inspired by Google's PageRank algorithm. The words in the text are regarded as nodes in the graph, and the co-occurrence relationship between words is regarded as the edge between nodes. Through iterative calculation of the importance score of nodes, the most important words in the text are finally identified. In specific implementation, first, a sliding window (usually 5 words) is set based on the segmentation results, and undirected edges are connected between the words in the window. Then, each node in the graph is assigned an initial weight, and the node weight is updated through multiple iterations until convergence or the preset number of iterations is reached. In this process, the words with more co-occurrence relationships with important words will get higher weight scores. Finally, the words are sorted according to their weight scores, and the top several words (usually 10-20) are selected as the keyword set of the public text. In order to further improve the accuracy of keyword extraction, the structural characteristics of the public text are also considered, and the words in important positions such as titles, abstracts, and chapter titles are given higher initial weights, so that the algorithm pays more attention to these words. At the same time, the part-of-speech factor is also considered, and nouns, proper nouns and other words that can better represent the theme are preferred as keywords. Through this TextRank algorithm combined with text structure and language characteristics, the core keywords of each public text can be accurately extracted, and these keyword sets highly summarize the theme and key points of the public text, providing refined semantic representation for subsequent feature construction.

[0098] In the specific implementation of the TextRank algorithm, the following key parameters are set: First, the sliding window size is set to 5, i.e. undirected edges are connected between the words in the window; for the words in the window, the weight of the edge is calculated according to the distance of the words in the window, the closer the distance, the greater the weight, and the reciprocal of the distance is used as the weight. After constructing the word co-occurrence graph, the initial weight of each node is set to 1.0, and then the node weight is updated through iteration. In the iterative calculation, the typical PageRank formula is adopted, which includes damping coefficient (set to 0.85), current node weight and the sum of weights contributed by adjacent nodes. The maximum number of iterations is set to 100, or when the weight change between adjacent iterations is less than 0.0001, it is considered to be converged. After the algorithm converges, the nodes are sorted according to their final weights, and the top 10-15 words with the highest weights are selected as keywords, and the specific number is dynamically adjusted according to the length of the text, usually calculated by the formula.

[0099] In addition, two improvements are made to the standard TextRank algorithm: first, the position information of the words is weighted, and the initial weight of the words in the title, the first paragraph and the last paragraph is set to 1.5, while the initial weight of the words in the body is 1.0; second, the part-of-speech information is combined to give higher weight (multiplied by 1.2) to nouns, verbs, adjectives and unknown words, while reducing the weight (multiplied by 0.8) of function words and other less contributing parts of speech.

[0100] Step S2.3: Splicing the keyword set with the title of the pre-processed multi-source public science and technology text data set, converting to vector representation using word embedding technology to obtain the enhanced text features. This sub-step is a key link of feature construction. First, the keyword set extracted in the previous step is spliced with the original title of the public text to form an enhanced text representation. This splicing strategy is based on two important considerations: on the one hand, the title of the public text usually summarizes the theme and purpose of the document and contains the most core information; on the other hand, the keywords extracted from the body of the text supplement more detailed content points and professional terms. By splicing the two, the summarization of the title and the details of the keywords can be retained, forming a more comprehensive text representation. After splicing, advanced word embedding technology is used to convert the text into a vector representation. Specifically, pre-trained Word2Vec, GloVe or BERT models are used to map each word to a fixed-dimensional dense vector (usually 300 or 768 dimensions), and then all word vectors are combined into a document vector through weighted averaging or other methods to serve as the semantic representation of the public text. In the vector construction process, the importance weight of the words is also considered, such as assigning different weights to different words based on TF-IDF values, so that more recognizable words have a greater impact in the final vector. In addition, metadata features of the public text, such as publication time (converted to relative time features), publishing agency (converted to agency type and level features), etc., are combined with the text semantic vector to form a multi-dimensional feature representation. This enhanced feature representation that combines text semantics and metadata can comprehensively capture the content characteristics and context information of the public text, providing rich and effective feature input for the subsequent classification model, significantly improving the model's understanding of the semantic of the public text and the accuracy of the classification.

[0101] In an embodiment of the present application, step S3 includes constructing a multi-dimensional public text classification system including talent support, financial support, technical support, and regulatory control and performing data labeling to establish a training data set, including:

[0102] Step S3.1: Design the multi-dimensional public text classification system including the public text tool dimension, the public text field dimension, and the public text level dimension, and formulate classification standards and judgment rules. This sub-step is the core work of building a public text classification system for science and technology. Based on the analysis of a large number of public texts for science and technology and the actual application requirements, a classification system of three key dimensions is designed. First, the public text tool dimension reflects the ways and means of supporting public texts, including talent support (such as talent introduction plan, scientific research personnel incentive policy, etc.), financial support (such as scientific research fund support, innovation project subsidy, etc.), technical support (such as technology transfer service, scientific research facility opening, etc.), and regulations and control (such as industry standard formulation, market access specification, etc.). Second, the public text field dimension describes the science and technology fields and industrial directions of public texts, including artificial intelligence, biomedicine, new materials, new energy, information technology, and other subdivided fields, as well as comprehensive policies across fields. Third, the public text level dimension reflects the publishing subject and applicable scope of public texts, including different administrative levels. For the classification system of these three dimensions, detailed classification standards and judgment rules are formulated, including the definition, scope, typical characteristics and boundary conditions of each category. For example, for the financial support category of public texts, it is stipulated that specific amount of funds, support methods, application conditions and other elements are required to be included; for the public texts in the field of artificial intelligence, it is stipulated that key technologies or applications such as intelligent algorithms, machine learning, computer vision, etc. are required to be mentioned. These classification standards and judgment rules form a complete classification guide document, which provides clear operation basis for subsequent data labeling, ensures the consistency and accuracy of the labeling process, and also provides a clear public text retrieval and screening framework for end users.

[0103] Step S3.2: Select public text samples using stratified sampling method, organize field experts to manually label according to the classification standard and the judgment rule, ensure the labeling quality through multi-person cross-labeling, and obtain the training data set. After completing the design of the classification system, the data labeling work is carried out. First, representative public text samples are selected from the pre-processed data set using stratified sampling method. Stratified sampling ensures the balanced distribution of samples in different periods, different publishing agencies and different content types, avoiding sample selection bias. Specifically, according to the publishing time, publishing agency and preliminary content analysis of public text, the data set is divided into multiple layers, and then samples are randomly selected from each layer in proportion. Finally, about 5000 representative public texts are selected as labeling samples. Next, a multi-disciplinary labeling team including policy research experts, science and technology management personnel and field technical experts is organized to manually label the selected samples. The labeling process strictly follows the classification standard and judgment rule formulated in the foregoing. Each labeling personnel needs to select appropriate class labels in three dimensions for each public text and record the judgment basis. In order to ensure the labeling quality, multi-person cross-labeling is adopted, that is, each public text is independently labeled by at least two labeling personnel, and then the consistency index (such as Cohen's Kappa coefficient) is calculated. For samples with high consistency, the majority labeling result is directly adopted; for samples with large differences, expert discussion meeting is organized, and consensus is reached through in-depth analysis and discussion. In addition, regular labeling verification link is set, and labeled samples are randomly selected for review, so that possible labeling errors can be found and corrected in time. Through a series of strict labeling process and quality control measures, a high-quality training data set is finally formed, which provides reliable supervision signal for subsequent model training and lays a solid foundation for the practical application of the classification system.

[0104] In an embodiment of the present application, in step S4, the training data set is subjected to feature extraction and field adaptability fine-tuning using the BERT pre-training model, and a public text classification model is obtained through parameter optimization and ensemble learning, comprising:

[0105] Step S4.1: Divide the training dataset into training set and test set, and perform BERT tokenizer processing on the text data of the training dataset, and expand the training samples through data augmentation operation. Before starting the model training, this sub-step first performs scientific division and preprocessing on the labeled training dataset. A stratified random sampling method is used to divide the dataset into training set and test set in a ratio of 8:2, ensuring that the distribution of the two subsets in each category remains consistent, avoiding evaluation bias caused by uneven data distribution. Time factor is also considered during division to ensure that the test set contains newer public text samples to better evaluate the model's performance in handling newly released public texts. After data division, the text data is processed using the BERT dedicated tokenizer. The BERT tokenizer uses the WordPiece algorithm, which can decompose text into sub-word units, effectively handling rare words and compound words, and is particularly suitable for processing professional terms and policy expressions in scientific and technological public texts. Tokenization processing converts text into the input format required by the model, including token IDs (token number), attention mask (attention mask, marking which positions are valid text), and segment IDs (segment number, used to distinguish different parts of text), etc. In order to enhance the generalization ability and robustness of the model, various data augmentation operations are also implemented. These operations include synonym replacement (using a synonym dictionary to randomly replace some words in the text), back-translation (translating the text into English and then back into Chinese, maintaining the semantics while changing the expression method), random insertion / deletion / swap (making minor changes to the text structure while maintaining the semantics), etc. These data augmentation techniques effectively expand the diversity of training samples, enabling the model to learn more diverse language expression variants and improving the model's ability to handle various expression methods. Especially for categories with fewer samples, data augmentation is particularly helpful in alleviating the problem of data imbalance. Through these careful data preparation work, high-quality and diverse training data are provided for subsequent model training, laying a foundation for the high performance and strong generalization ability of the model.

[0106] Step S4.2: Domain adaptation fine-tuning based on Chinese BERT model, setting parameters including learning rate, batch size, using cross-entropy loss function and Adam optimizer for model training. This sub-step is the core link of model training, and the pre-trained Chinese BERT model (such as bert-base-chinese) is selected as the basic model, and the domain adaptation fine-tuning is carried out according to the characteristics of the public science and technology text. The BERT model adopts the Transformer architecture, and can capture the long-distance dependence relationship in the text through the self-attention mechanism, which is particularly suitable for processing the complex semantic structure in the public science and technology text. In the model architecture design, an output layer suitable for multi-label classification is added on the basis of the BERT encoder, including three independent classification heads corresponding to the classification tasks of the public text tool dimension, the field dimension and the level dimension. In order to optimize the model training effect, a series of key parameters are carefully set. First, a small initial learning rate (usually 2e-5 to 5e-5) is used, combined with learning rate warm-up and linear decay strategy, which slowly increases the learning rate in the early stage of training, and then gradually reduces with the training, avoiding unstable training or falling into local optimum. Secondly, according to the GPU memory limit and training efficiency, the appropriate batch size (batch size, usually 16 or 32) is set, which maximizes the use of computing resources while ensuring the stability of training. In addition, appropriate training rounds (epochs, usually 3 to 5 rounds) are set, and early stopping strategy is implemented, that is, when the performance of the validation set does not improve continuously for many rounds, the training is automatically stopped to prevent overfitting. In the selection of loss function, for each classification dimension, the cross-entropy loss function is used, which can effectively measure the difference between the predicted distribution and the real label distribution, and is particularly suitable for multi-class classification problems. In the aspect of optimizer, Adam optimizer is selected, which combines the advantages of momentum method and adaptive learning rate method, can automatically adjust the learning rate of each parameter, accelerate convergence and improve training stability. Through these careful model design and parameter optimization, the BERT model is effectively migrated to the field of public science and technology text, so that the model can accurately understand and classify various public science and technology texts.

[0107] In the model architecture, the present application adopts the bert-base-chinese pre-training model, which contains 12 Transformer encoder layers, each layer has 12 attention heads, and the hidden layer dimension is 768. The total parameter quantity of the model is about 110 million. In the model structure design, three independent classification heads are added on the top of the BERT encoder, which are responsible for the classification tasks of the public text tool dimension, the field dimension and the level dimension respectively. Each classification head is composed of a linear layer (768-dimensional input, class number output) and a softmax activation function.

[0108] In terms of training hyperparameter settings, the following specific configurations are adopted: the initial learning rate is set to 3e-5, combined with a linear warm-up strategy (warm-up step number is 1000) and a linear decay strategy; the batch size is set to 32; the training rounds are 4 rounds, but an early stopping mechanism is set, which automatically stops training when the performance on the validation set does not improve for 2 consecutive rounds; the weight decay coefficient is set to 0.01 to alleviate overfitting; the gradient clipping threshold is set to 1.0 to prevent gradient explosion. During the training process, the cross-entropy loss function is used, and the Adam optimizer is used, where the first momentum coefficient is 0.9, the second momentum coefficient is 0.999, and the numerical stability constant is 1e-8.

[0109] In terms of training process monitoring, the change trend of training loss and validation loss is recorded. Generally, the training loss gradually decreases from about 2.1 initially to about 0.3, and the validation loss decreases from about 1.9 initially to about 0.4, and then tends to be stable. In terms of performance indicators on the validation set, after the model training is completed, the average accuracy on each classification dimension reaches 92.7%, and the F1 value reaches 91.3%, which is significantly better than traditional machine learning methods (such as the 87.1% accuracy of SVM) and ordinary deep learning methods (such as the 89.4% accuracy of TextCNN).

[0110] Step S4.3: Use ensemble learning method to combine the prediction results of multiple models to improve classification accuracy and obtain the public text classification model. After the training of the basic model is completed, this sub-step further improves the classification performance through ensemble learning technology. The core idea of ensemble learning is to combine the prediction results of multiple basic models, reduce the bias and variance of individual models through "collecting ideas from all walks of life", and improve the overall prediction accuracy and stability. Three ensemble learning strategies are mainly used: voting method, stacking and model fusion. First, the voting method is the most direct integration method, and multiple BERT models with similar structures but different initializations or different hyperparameters are trained, and then the classification results of each sample are voted, and the class with the most votes is selected as the final prediction. This method is simple and effective, and can reduce the random error of individual models. Second, stacking is a more complex ensemble method, which first trains multiple basic models of different types, including BERT, RoBERTa, ALBERT and other different pre-training models, as well as CNN, BiLSTM and other traditional deep learning models. The prediction results of these basic models are used as features and input into a meta-learner (usually logistic regression or lightweight neural network) for secondary learning to generate the final prediction result. This method can effectively combine the advantages of different models and capture more complex patterns. Third, the model fusion technology obtains a smoother and more reliable prediction distribution by weightedly averaging the prediction probabilities of multiple models (rather than hard voting of class labels). The weights can be dynamically allocated based on the performance of each model on the validation set, so that the models with better performance obtain higher weights. Through the comprehensive application of these ensemble learning methods, the classification accuracy and robustness are significantly improved, especially when dealing with ambiguous boundaries or complex public texts. The final public text classification model not only achieves excellent performance indicators (such as accuracy, F1 value, etc.) on the test set, but also exhibits good generalization ability and stability in practical applications, and can accurately classify various newly released public texts in science and technology, providing a reliable foundation for subsequent analysis and recommendation services.

[0111] In an embodiment of the present application, the feature extraction and domain adaptation fine-tuning using the BERT pre-training model comprises:

[0112] Step S4.4: Based on the pre-processed multi-source scientific public text dataset, a multi-dimensional feature space containing semantic correlation, time correlation and institution correlation is constructed, and semantic vectors are calculated and public text influence is evaluated through word embedding method. In this embodiment, not only the semantic content of public text is considered, but also multi-dimensional features such as time and institution are integrated to construct a more comprehensive feature space. For semantic correlation, word embedding technologies such as Word2Vec and Doc2Vec are used to convert public text content into high-dimensional vector representation and capture the semantic information of the text; for time correlation, a time decay function is designed to give higher weight to recently published public texts, reflecting the timeliness of public texts; for institution correlation, an authority scoring system for publishing institutions is established, and different levels and types of public service department institutions are given different weights. Through the combination of these multi-dimensional features, the influence of public text can be comprehensively evaluated, and more rich feature representation is provided for subsequent classification and recommendation. In addition, a feature fusion mechanism is realized, which automatically learns the importance of different dimensional features through attention network, and dynamically adjusts the weight of each dimension according to the specific task, so that the feature representation is more adaptive to different application scenarios.

[0113] Step S4.5: An incremental learning strategy is adopted to make the public text classification model continuously learn new public text information, an anomaly detection algorithm is used to identify emerging concepts to dynamically expand the feature space, and a public text knowledge graph is established to realize automatic expansion of concept boundaries. Considering the continuous updating and evolution of knowledge in the field of scientific public text, the present application adopts an incremental learning strategy, which enables the model to continuously learn new public text information without retraining. Specifically, new published public texts are collected regularly and compared with the prediction results of the existing model. For samples with low prediction confidence, they will be marked as potential new concepts or new categories. Then, an anomaly detection algorithm (such as Isolation Forest, One-Class SVM, etc.) is used to further verify whether these samples indeed represent new knowledge points. Once confirmed, the feature space will be dynamically expanded to create new representations for these new concepts. At the same time, a public text knowledge graph is constructed, which identifies entities and extracts relationships to represent key concepts in public texts and their relationships as a graph structure, and integrates these knowledge into the model through graph embedding technology. The introduction of knowledge graph enables automatic expansion of concept boundaries and captures the relationship between concepts, greatly enhancing the model's understanding ability of emerging fields and cross-disciplinary public texts.

[0114] Step S4.6: Define multiple optimization objectives including classification accuracy, class balance, and diversity, use multi-objective optimization algorithms to find the balance points of each objective, and get the multi-objective optimized public text classification model. Traditional classification models usually only focus on a single objective of classification accuracy, but in practical applications, factors such as class balance and diversity are also important. Therefore, the invention defines three key optimization objectives: classification accuracy (measured by accuracy, recall, F1 value, etc.), class balance (ensure that the classification performance of each class is similar, avoid ignoring some classes), and diversity (ensure that the model can recognize and process diverse public text expressions). In order to optimize these three possibly conflicting objectives at the same time, multi-objective optimization algorithms such as NSGA-II (Non-dominated Sorting Genetic Algorithm II) and MOEA / D (Decomposition-based Multi-objective Evolutionary Algorithm) are used. These algorithms use evolutionary computation to find the Pareto optimal solution set in the objective space, that is, those solutions that cannot be further improved without sacrificing one objective. From these Pareto optimal solutions, according to the needs of specific application scenarios, the most suitable balance point is selected as the final model. This multi-objective optimization method enables the public text classification model to maintain high accuracy while also fairly handling each type of public text and adapting to diverse expression methods, greatly improving the practicality and robustness of the model in actual applications.

[0115] In an embodiment of the present application, the multi-objective quality diversity optimization further comprises:

[0116] Step S4.7: Introducing diversity balancing strategies to ensure that the personalized public text recommendation results cover different types of public text information, and designing diversity reward mechanisms to encourage the multi-objective optimized public text classification model to explore public text domains that the user has not contacted. To avoid the common "information cocoon" problem in recommendations, the invention introduces diversity balancing strategies to ensure that the public texts recommended to the user are not limited to their known interest areas. Specifically, a deterministic point process (DPP) model is used, which can naturally promote the diversity of the result set while ensuring relevance. The similarity matrix between candidate public texts is calculated, and a DPP algorithm is used to select a subset of relevant and diverse public texts as the final recommendation result. In addition, a diversity reward mechanism is designed, which gives the model an additional reward when it recommends public text domains that the user has not contacted and the user has positive feedback on these recommendations, encouraging the model to continue exploring the user's potential interest points. This mechanism is implemented through reinforcement learning, and the model will gradually learn the balance strategy of appropriately introducing novel content while ensuring basic recommendation quality. Through these diversity optimization measures, more comprehensive and balanced public text recommendations can be provided to users, helping users expand their knowledge and discover new public text domains of interest.

[0117] Step S4.8: Use attention mechanism to dynamically evaluate the importance weight of each feature in the multi-dimensional feature space, and establish a feature feedback loop to adjust the feature weight system through user interaction data. In the multi-dimensional feature space, the importance of different features to classification and recommendation often changes dynamically, depending on the specific task and user. To capture this dynamic nature, the invention uses an attention mechanism that can automatically adjust the importance weight of each feature according to the current input and task. Specifically, a multi-head self-attention structure is used, which allows the model to learn the relationship between features from multiple angles at the same time and assign appropriate weights to each feature. These weights are not fixed, but are dynamically calculated based on the input public text content and user context, allowing the model to flexibly adjust the importance of features for different situations. In addition, a feature feedback loop is established to continuously evaluate and adjust the feature weight system by collecting user interaction data (such as clicks, collections, shares, etc.) with the recommendation results. For example, if it is found that the user is particularly interested in public texts published by a certain institution, the weight of the institution-related feature will be increased; if the user tends to focus on newly published public texts, the importance of the time-related feature will be increased. This feature weight adjustment mechanism based on user feedback allows the feature representation to be continuously optimized, providing more personalized and accurate public text classification and recommendation services.

[0118] Step S4.9: Extend the traditional evaluation indicators to introduce diversity indicators including coverage, novelty, and build a diversity-aware evaluation system. Traditional recommendation evaluation mainly focuses on accuracy indicators such as precision, recall, and F1 value, etc., but these indicators cannot fully reflect the actual effect of recommendation, especially in terms of diversity and user satisfaction. Therefore, the invention extends the evaluation system and introduces a series of diversity-related indicators: coverage (Coverage, measures the proportion of recommended results covering public text categories), novelty (Novelty, measures the proportion of content in the recommended results that users have not encountered), serendipity (Serendipity, measures the proportion of content in the recommended results that surprises users), and diversity (Diversity, measures the degree of difference within the recommended results). These indicators together constitute a diversity-aware evaluation system, which can evaluate the performance of recommendation from multiple dimensions. In addition, user satisfaction surveys and A / B testing schemes are designed to collect user feedback directly and compare the actual effects of different algorithms, further verifying and optimizing the recommendation strategy. In the long run, the expansion of users' knowledge will also be tracked to evaluate the contribution of recommendation to promoting users' understanding of a wider range of public text fields. This comprehensive evaluation system enables the continuous optimization of diversity and user experience while ensuring basic recommendation accuracy, providing more balanced and valuable public text recommendation services.

[0119] In one embodiment of the present application, the following steps are further included:

[0120] When using the BERT pre-training model for feature extraction and domain adaptation fine-tuning, the LPLC2 neural network is fused to realize multi-target detection of public text content, including:

[0121] Step S4.10: The pre-processed multi-source scientific public text data set is divided into text blocks by sentence and paragraph, the text blocks are converted into high-dimensional vectors by using BERT word embedding technology, the semantic similarity is reserved by mapping the high-dimensional vectors to two-dimensional space through dimension reduction algorithm, a two-dimensional semantic feature map is constructed, and two-dimensional representation conversion of the public text is realized. Traditional text processing methods usually regard text as a one-dimensional sequence, which is difficult to capture the complex structure and relationship inside the text. In order to solve this problem, the present application proposes an innovative two-dimensional representation conversion method, which converts the public text document into a two-dimensional feature map similar to an image, so that the LPLC2 network originally designed for image processing can be applied to text analysis. In specific implementation, first, the public text is divided into multiple text blocks according to natural paragraphs and sentences, and each text block represents a semantic unit in the document. Then, the BERT model is used to encode each text block to obtain a 768-dimensional high-dimensional vector representation, which captures the deep semantic information of the text block. Next, the t-SNE or UMAP dimension reduction algorithm is used to map these high-dimensional vectors to a two-dimensional space while preserving the semantic similarity relationship between the vectors. In this two-dimensional space, semantically similar text blocks will be mapped to similar positions, forming semantic clusters. Finally, based on these two-dimensional coordinates, a two-dimensional semantic feature map is constructed, in which each "pixel point" corresponds to a text block, and the "color" (i.e. feature value) of the text block is determined by the semantic feature of the text block. This two-dimensional representation method not only preserves the semantic information of the text, but also reflects the relative position and relationship between the text blocks, providing an ideal input format for subsequent multi-target detection.

[0122] Step S4.11: define key element types including public text target, implementation subject, support measures, constraint conditions, design multi-element labeling specification, develop auxiliary labeling tool to support text area labeling, and establish fine-grained public text multi-element labeling system. In order to realize the fine analysis of public text content, the application defines a series of key element types, including public text target (the problem to be solved or the goal to be achieved by public text), implementation subject (the institution or organization responsible for the implementation of public text), support measures (specific support methods and resources provided by public text), constraint conditions (limiting conditions and requirements for the implementation of public text), etc. Based on these element types, detailed multi-element labeling specification is designed, which clearly defines the judgment standard, boundary definition and labeling format of each type of element. In order to improve the labeling efficiency and accuracy, a special auxiliary labeling tool is developed, which provides a visual interface to support the labeling personnel to select the area directly on the text and assign the category label, while providing real-time labeling suggestions and consistency check. In the labeling process, multi-person cross-labeling method is adopted, that is, the same document is independently labeled by multiple labeling personnel, and then the labeling quality is evaluated by calculating the consistency index (such as Cohen's Kappa coefficient), and the samples with low consistency are discussed and corrected. Through this fine-grained multi-element labeling system, high-quality training data can be obtained to provide reliable supervision signal for the subsequent multi-target detection model, and accurate understanding and positioning of public text content can be realized.

[0123] Step S4.12: Extract the contextual semantic representation of the public text using the BERT model, convert the BERT hidden layer features into a two-dimensional feature map as the input of the LPLC2 network, apply the local pyramid and contrast modules of LPLC2 to enhance the key element boundary recognition, design a multi-task learning head to output the overall classification and element detection results simultaneously, and obtain the LPLC2-BERT fusion public text classification model. The present application innovatively fuses the BERT model with the LPLC2 network, combining the advantages of both, and realizes the unification of overall classification and internal element accurate detection of public text. In terms of model architecture, the BERT model is first used to process the original text input to extract rich contextual semantic representation. Then, the hidden layer features of each layer of BERT (especially the features of the middle layers, which contain rich grammatical and semantic information) are reorganized into a two-dimensional feature map as the input of the LPLC2 network. The LPLC2 network is a neural network structure specially designed for object detection, and its core components include a local pyramid module (to capture multi-scale features) and a local contrast module (to enhance the target boundary). These modules can effectively process two-dimensional feature maps to identify significant areas and boundaries within them, thereby accurately locating various key elements in public text. At the output end of the network, a multi-task learning head is designed to perform two tasks simultaneously: one is to classify the entire public text and output its public text category; the other is to detect and locate key elements in the text and output the type and position information (coordinates on the two-dimensional feature map) of each element. This LPLC2-BERT fusion model can simultaneously optimize classification and detection performance through end-to-end joint training, achieving comprehensive understanding and fine-grained analysis of public text content.

[0124] The specific architecture design of the LPLC2-BERT fusion network is as follows: First, the BERT encoder part adopts the standard bert-base-chinese model (12 layers of Transformer, 12 attention heads, 768-dimensional hidden layer), but only the first 8 layers are retained to extract basic semantic features. This can reduce the amount of calculation while retaining sufficient semantic information. The feature dimension of the BERT encoding is the sequence length multiplied by the 768-dimensional vector.

[0125] In order to convert the one-dimensional sequence features of BERT into the two-dimensional feature map required by LPLC2, a feature reorganization module is designed, which uses the following steps: First, rearrange the hidden states of the last layer of BERT into an approximately square two-dimensional feature map, with dimensions of batch size, height, width, and channel number, where the height and width are determined by the square root of the sequence length; then, reduce the channel dimension from 768 to 256 through 1x1 convolution to obtain a feature map with 256 channels.

[0126] The specific structure of the LPLC2 network includes:

[0127] 1. Local Pyramid Module: This module consists of three parallel residual branches, each containing convolution kernels of different sizes (3×3, 5×5, and 7×7) to capture features at different scales. Each branch includes a convolutional layer, a batch normalization layer, and a ReLU activation function, which are then fused through addition and residually connected to the input features.

[0128] 2. Local Contrast Module: First, average pooling and max pooling are applied to the input features to obtain the channel average and maximum, respectively. These two features are then concatenated and subjected to dimensionality reduction through convolution. A sigmoid function is then used to generate a spatial attention map. Finally, the attention map is multiplied with the original features to enhance the contrast of salient areas.

[0129] 3. Feature enhancement path: This path consists of four groups of alternating local pyramid modules and local contrast modules, forming a deep feature extraction network. Each group is followed by a feature downsampling layer (2×2 max pooling) to expand the receptive field.

[0130] 4. Feature Pyramid Network: This network uses a top-down feature fusion approach to integrate feature maps of different scales. The specific structure includes a top-down upsampling path (using 2× upsampling) and lateral connections (using 1×1 convolution for channel alignment). The network outputs feature maps of four different scales, with the resolution halved sequentially.

[0131] 5. Multi-task learning head: contains two parallel task branches:

[0132] a) Classification branch: The feature map is converted into a vector through global average pooling, and then the common text classification result is output through a fully connected layer;

[0133] b) Detection branch: Apply convolution to the feature maps at each scale, and output the target position prediction (4 coordinate values) and category prediction (number of categories plus 1, including background class).

[0134] The total number of parameters in this fusion network is approximately 140 million, of which approximately 90 million are for BERT and approximately 50 million for LPLC2. The forward inference time on a GPU (such as a Tesla V100) is approximately 120 milliseconds per sample.

[0135] In one embodiment of the present invention, the LPLC2-BERT fusion public text classification model further includes attention mechanism enhancement and joint optimization, including:

[0136] Step S4.13: Introduce self-attention mechanism in BERT encoding layer to capture long-distance dependencies between public text words, design cross-modal attention module to make LPLC2 network focus on key areas of feature map according to semantic importance, realize combination of channel attention and spatial attention to enhance feature representation, construct hierarchical attention fusion mechanism to integrate features at different levels, realize attention-enhanced feature extraction. In order to further improve the feature extraction capability of the model, the application introduces a multi-level attention mechanism in the LPLC2-BERT fusion model. First, the original self-attention mechanism in the BERT encoding layer is strengthened, so that the model can better capture the long-distance dependencies between words in the public text, which is particularly important for understanding complex public text language. Secondly, an innovative cross-modal attention module is designed, which can use the semantic information extracted by BERT as a guide to help the LPLC2 network focus on the key areas of the feature map. Specifically, the semantic representation of BERT is used to generate an attention weight map, and then these weights are applied to the feature map of LPLC2 to realize semantic-guided visual attention. In addition, channel attention (emphasizing important feature channels) and spatial attention (emphasizing important spatial positions) are combined, and through the Squeeze-and-Excitation network and the spatial attention module, the discriminability of the feature representation is enhanced from the channel dimension and the spatial dimension respectively. Finally, a hierarchical attention fusion mechanism is constructed to weight and fuse features at different levels from BERT and LPLC2, forming a multi-scale and multi-level comprehensive feature representation. This multi-level attention enhancement mechanism enables the model to more accurately identify key information and important elements in the public text, significantly improving the quality and efficiency of feature extraction.

[0137] Step S4.14: Construct a joint loss function including public text classification loss and element detection loss, introduce consistency loss between BERT representation and LPLC2 features, design a dynamic weight balancing strategy to adjust the proportion of each loss contribution, and realize multi-task joint optimization training. In order to effectively train the LPLC2-BERT fusion model, the present application designs a complex joint loss function, which contains multiple components to guide the optimization direction of the model. First, the public text classification loss adopts the cross-entropy loss function to evaluate the prediction accuracy of the model for the overall public text category. Second, the element detection loss includes two parts: the bounding box regression loss (uses smooth L1 loss to measure the gap between the predicted boundary and the true boundary) and the element category classification loss (uses cross-entropy loss to evaluate the accuracy of element type prediction). In addition, in order to ensure the consistency between the semantic features extracted by BERT and the visual features processed by LPLC2, a feature consistency loss is introduced to minimize the distance (such as cosine distance or KL divergence) between the two kinds of feature representations, promoting the alignment and complementarity of information between modalities. Considering that the importance of different loss components may change with the training process and specific tasks, a dynamic weight balancing strategy is designed to automatically adjust the weight coefficients of each component according to the gradient size, training progress and validation performance. For example, in the early stage of training, more attention may be paid to the basic classification task, while gradually increasing the weight of element detection and feature consistency as the training progresses. This multi-task joint optimization training method enables the model to learn multiple related tasks simultaneously, not only improving the overall performance, but also enhancing the generalization ability and robustness of the model.

[0138] In the joint training of LPLC2-BERT fusion model, a carefully designed multi-task learning strategy and loss function are adopted. The joint loss function is composed of three main parts: classification loss, detection loss and feature consistency loss, which are weighted and summed through weight coefficients.

[0139] The classification loss adopts the cross-entropy loss function to calculate the difference between the predicted class probability and the true class label. The detection loss includes two parts: bounding box regression loss and element category classification loss. The bounding box regression loss adopts smooth L1 loss to calculate the gap between the predicted bounding box and the true bounding box; the element category classification loss adopts focal loss to alleviate the class imbalance problem, where the focus parameter is set to 2.0. The feature consistency loss calculates the cosine similarity between BERT features and LPLC2 features to ensure the semantic consistency of the two kinds of feature representations.

[0140] The weight coefficient adopts a dynamic adjustment strategy, the initial value is set as 1.0 of the classification loss weight, 0.5 of the detection loss weight and 0.1 of the feature consistency loss weight, and then is dynamically adjusted along with the training process: focusing on the learning of the basic classification task in the early stage of training; gradually increasing the weight of the detection task in the middle stage of training; increasing the weight of the feature consistency in the later stage of training.

[0141] In addition, a gradient balance mechanism is also designed to monitor the gradient norm of each task loss, and when the gradient of a certain task is obviously greater than that of other tasks (more than 3 times), the gradient of the task is automatically scaled to ensure the balance of learning of each task.

[0142] In the optimizer selection, the AdamW optimizer with weight decay is adopted, the learning rate of the BERT part is set to 1e-5, the learning rate of the LPLC2 part is set to 1e-4, and the weight decay is 0.01. The learning rate scheduling adopts the cosine annealing strategy with a 10% round preheating stage.

[0143] Step S4.15: Implement model quantization and pruning to reduce inference time, develop a non-maximum suppression algorithm to filter redundant detection results, construct a logical association graph between detection elements, and obtain a structured public text understanding representation and an optimized public text classification model. In order to improve the efficiency and practicality of the model in actual application, a series of inference optimization and post-processing techniques are implemented. First, model quantization and pruning techniques are used to reduce model size and computational complexity, including weight quantization (converting 32-bit floating-point numbers to 8-bit integers), structural pruning (removing unimportant neuron connections) and knowledge distillation (transferring knowledge from large models to small models). These techniques significantly reduce the storage requirements and inference time of the model, enabling it to run efficiently in resource-limited environments. Second, a non-maximum suppression (NMS) algorithm suitable for text element detection is developed to filter redundant detection results. When multiple detection boxes overlap, the algorithm retains the one with the highest confidence and suppresses the other overlapping boxes, ensuring that each element is detected only once. In addition, a logical association graph between detection elements is constructed to analyze the positional relationships, semantic associations and contextual dependencies between elements, and infer their logical structures such as inclusion, order and causality. This structured public text understanding representation not only provides information about the location and type of elements, but also reveals their deep logical relationships, enabling a more comprehensive and in-depth understanding of public text content, providing a solid foundation for subsequent public text analysis and intelligent services.

[0144] In an embodiment of the present application, in step S5, the newly added multi-source scientific and technological public text data to be classified is automatically classified based on the public text classification model to obtain a classification result; and data mining techniques are used to analyze the distribution characteristics of public text, and visual tools are used to generate analysis result charts, including:

[0145] Step S5.1: Based on the common text classification model, automatically classify the newly added multi-source scientific and technological public text data to obtain the classification result. After the model training is completed, this sub-step applies the trained classification model to the actual scene to automatically classify the newly collected scientific and technological public text data. First, the same preprocessing procedures as the training data are performed on the new data, including text cleaning, format unification, and feature extraction steps, to ensure that the data format input into the model is consistent with that during training. Then, the processed data is input into the integrated classification model, which will make predictions in three dimensions (tool dimension, field dimension, and level dimension) and output the probability distribution of each category. According to the preset probability threshold (usually 0.5 or an empirical value), the final classification label is determined. For multiple categories with close probabilities, multiple labels can be assigned simultaneously, reflecting the multi-class characteristics of public text. In particular, for "difficult samples" with low prediction probabilities or close probabilities of multiple categories, they are automatically marked as "need manual review" and submitted to professional personnel for review to ensure the reliability of the classification results. In addition, a confidence evaluation mechanism for classification results is also implemented, which calculates the confidence score based on the distribution characteristics of the prediction probability to provide a reference for subsequent analysis and application. To cope with the dynamic changes of public text content and expression, a regular update mechanism for the classification model is also designed, which collects new labeled data every certain period of time (such as every quarter) to fine-tune the model, ensuring that the model can adapt to the latest characteristics and expression methods of public text. Through this combination of automatic classification and manual review, as well as regular updating of the classification mechanism, a large amount of newly added scientific and technological public text data can be efficiently and accurately processed, providing a solid foundation for subsequent data analysis and intelligent services.

[0146] Step S5.2: Analyze the distribution characteristics of public texts using data mining techniques. After the automatic classification of public texts is completed, this sub-step uses advanced data mining techniques to analyze the classification results in multiple dimensions to uncover the distribution characteristics and evolution patterns of public texts. First, trend analysis in the time dimension is performed. Time series analysis methods are used to examine the trend of the number of various types of public texts, identify periodic fluctuations, sudden growth, and long-term evolution trends. For example, it can be found that certain types of public texts (such as technology innovation funding support) are periodically released at specific time periods (such as the beginning of the year or after the budget) and that the rapid growth trend of public texts related to a certain technology field (such as artificial intelligence) can be captured. Second, regional analysis in the spatial dimension is implemented. The type distribution and density differences of public texts in different regions (such as different provinces and different city groups) are compared, and regional public text heat maps are generated to visually display the support intensity and focus of public texts in each region. This analysis helps to identify the differences and competition patterns of public texts among regions, providing reference for enterprise location selection and regional innovation research. In addition, content dimension correlation analysis is also conducted. Through association rule mining and network analysis methods, the correlation patterns and co-occurrence relationships between different types of public texts are explored, the public text correlation network is constructed, and the matching relationship and policy combination mode between different public text tools are revealed. For example, it can be found that there is a high correlation between talent introduction policies and scientific research funding support policies, or that certain types of public texts usually use specific support mode combinations. These multi-dimensional data mining analyses not only reveal the distribution patterns and evolution trends of public texts, but also provide valuable data insights for decision-makers, helping them understand the overall pattern and development dynamics of public text support, and providing data support for the formulation and adjustment of science and technology innovation policies.

[0147] Step S5.3: Generating analysis result charts through visualization tools. After completing the data mining analysis, this sub-step utilizes advanced data visualization techniques to transform complex analysis results into intuitive and easy-to-understand chart forms, improving data readability and decision-making reference value. A variety of visualization tools and technologies are adopted, including Web-based interactive visualization libraries (such as Echarts, D3.js) and professional data visualization platforms (such as Tableau, PowerBI), and the most appropriate visualization method is selected according to the characteristics of different analysis results. First, for time trend analysis, dynamic time series charts are generated, including line charts, area charts, and heat calendar charts, which intuitively display the trend and periodic characteristics of the number of public texts over time. These charts support multi-dimensional filtering and comparison, and users can simultaneously view the time evolution comparison of different types or different regions of public texts. Second, for spatial distribution analysis, interactive geographic information visualization is created, including regional heat maps, bubble maps, and hierarchical maps, which show the distribution density and type characteristics of public texts in different geographic regions. These maps support multi-level drilling analysis, and users can gradually drill down from the national view to the provincial, municipal, and even district-level detailed view. In addition, for public text content and association analysis, network relationship diagrams, Sankey diagrams, and co-occurrence matrices are designed to show the association strength and flow relationship between different types of public texts. In particular, the network relationship diagram can intuitively reflect the core nodes and key connections of the public text network through visual encoding of nodes and edges (such as size, color, thickness, etc.). To improve user experience and analysis flexibility, interactive data panels are also developed to support multi-dimensional data filtering, comparison, and exploration. Users can select specific time ranges, regions, public text types, and other conditions according to their needs to generate customized analysis views. These visualization charts not only support internal display but also support export in various formats (such as PNG, PDF, interactive HTML, etc.) for easy use in reports and demonstrations. Through these rich and diverse visualizations, complex data analysis results are transformed into intuitive and clear visual information, greatly improving data understandability and usability, and providing strong visual support for science and technology innovation decision-making.

[0148] In an embodiment of the present application, in step S6, a user portrait is constructed based on user interest data including user registration information, browsing history, and search keyword data; a public text feature index is constructed based on the classification results and the enhanced text features; and a recommendation algorithm is used to calculate the similarity between user interest and public text features based on the user portrait and the public text feature index, to generate personalized public text recommendation results, including:

[0149] Step S6.1: Based on user interest data including user registration information, browsing history and search keyword data, build user portrait. This sub-step is the basic link of personalized recommendation service. Through multi-source data fusion, a comprehensive and accurate user interest portrait is constructed. First, collect and analyze user registration information, including basic attributes (such as industry, position, professional background, etc.) and self-filled interest tags. These information provides the initial framework of user interest. Second, continuously track and analyze user browsing behavior data, including viewed public text types, dwell time, collection and download behavior, etc. These implicit feedback data reflect the actual interest tendency and intensity of the user. Third, record and analyze user search keyword data. These explicit query information directly reflects the user's immediate information needs and focus. Based on data collection, multiple machine learning techniques are used to build user interest models. First, use collaborative filtering algorithm to identify interest similar user groups based on user behavior similarity, and infer individual possible interest points from group behavior. Second, apply content analysis technology to map user interaction public text content features to interest space and build content-based interest representation. Third, use topic model (such as LDA) to extract latent topics from user interaction text and form topic-level interest distribution. These different dimension interest representations are integrated into a unified user portrait, including long-term stable interest and short-term dynamic interest. Long-term interest reflects the user's basic professional direction and continuous attention area, which is obtained through long-term behavior data accumulation and smoothing; short-term interest captures the user's immediate needs and attention shift, mainly based on recent interaction behavior and search records. Dynamic updating mechanism of user portrait is also realized, which continuously adjusts the interest model according to the latest behavior of the user, ensuring that the portrait can timely reflect the changes and development of user interest. Through this multi-source data fusion, multi-dimensional modeling user portrait construction method, the interest characteristics and information needs of the user can be comprehensively and accurately understood, providing reliable user-side data support for subsequent personalized recommendation.

[0150] Step S6.2: Constructing a public text feature index based on the classification results and the enhanced text features. While completing the user portrait construction, this sub-step constructs an efficient feature index for public text data, providing data support for subsequent similarity matching and recommendation calculation. First, the classification results and enhanced text features in the previous steps are integrated to construct a multi-dimensional feature vector for each public text. These features include three-dimensional classification labels (tool dimension, domain dimension, and hierarchical dimension), text semantic vectors (document embedding based on BERT or Word2Vec), and metadata features (such as publication time, publishing organization, etc.). To support efficient similarity calculation and retrieval, a hybrid index structure combining inverted index and vector index is adopted. For structured features such as classification labels and metadata, a traditional inverted index is established to support exact matching and range queries; for high-dimensional features such as text semantic vectors, Approximate Nearest Neighbor (ANN) search techniques such as HNSW (Hierarchical Navigable Small World) or Annoy algorithm are used to construct an efficient vector index, supporting fast similarity search. This hybrid index structure ensures both the accuracy of queries and the efficiency of high-dimensional vector similarity calculation. To further improve the performance and flexibility of the index, a multi-level cache strategy and dynamic update mechanism are implemented. The multi-level cache caches the features of popular public texts and common query results in memory, significantly reducing disk IO and computational overhead; the dynamic update mechanism ensures that newly added public texts can be indexed in a timely manner, and maintains the index structure through incremental updates, avoiding the high cost of full reconstruction. In addition, a feature weight adaptive adjustment mechanism is designed to dynamically adjust the weight of different features in similarity calculation based on feedback data from different user groups, so that the index can better adapt to the needs of different scenarios. Through this multi-dimensional feature fusion and efficient index structure, the public text feature index provides strong content-side data support for subsequent personalized recommendation, ensuring the accuracy and efficiency of similarity calculation in the recommendation process.

[0151] Step S6.3: Based on the user profile and the public text feature index, a recommendation algorithm is used to calculate the similarity between user interest and public text features, and generate personalized public text recommendation results. After the user profile and the public text feature index are ready, this sub-step implements an intelligent recommendation algorithm to generate personalized science and technology public text recommendation results for users. A hybrid recommendation method combining multiple recommendation strategies is used to balance the relevance, diversity and novelty of the recommendations. First, content-based recommendation is the core strategy, which calculates the similarity (such as cosine similarity, Euclidean distance, etc.) between the user interest vector and the public text feature vector to identify the public text that best matches the user's interest. This calculation process considers multiple dimensions of feature matching, including theme relevance, tool type matching and application level compliance, and dynamically adjusts the weights of each dimension based on user historical feedback. Second, collaborative filtering recommendation is used as a supplementary strategy, based on the assumption that "users with similar interests may be interested in similar public texts". This method can discover implicit interest associations that are difficult to capture based on content, enriching the breadth of the recommendation results. In addition, a knowledge graph enhanced recommendation strategy is implemented, which uses the constructed public text knowledge graph to discover indirect associations between user interest and public text through graph algorithms (such as random walk, path reasoning, etc.), further improving the accuracy and explainability of the recommendations. After generating the preliminary recommendation list, multi-objective sorting optimization is performed. First, consider the timeliness factor, give higher sorting weight to newly released relevant public texts, ensure that users can timely access the latest policy information. Second, consider the diversity factor, through algorithms such as Determinantal Point Process (DPP), increase the diversity of the recommendation results while ensuring relevance, avoid too single recommendation results. Third, consider the personalization factor, adjust the recommendation sorting according to the user's specific attributes (such as the region, enterprise size, etc.), preferentially recommend public texts that are more suitable for the user's actual situation. Finally, a recommendation explanation generation module is designed to automatically generate personalized recommendation reasons for each recommendation result, such as "based on your interest in artificial intelligence" and "related to the scientific research funding policy you recently viewed", enhancing the transparency and user trust of the recommendations. Through this multi-strategy integration and multi-objective optimization recommendation method, users can be provided with highly personalized, content-rich and timely updated science and technology public text recommendation services, effectively meeting the diverse needs of users in the process of scientific and technological innovation for public text information.

[0152] To comprehensively evaluate the model performance, detailed tests and comparisons were conducted in multiple dimensions. On the public text classification task, the LPLC2-BERT fusion model significantly outperformed the existing technology. Compared with the traditional TF-IDF plus SVM method, the model improved the average F1 value by 10.6 percentage points (from 84.8% to 95.4%); compared with the base BERT model, it improved by 2.8 percentage points (from 92.6% to 95.4%). On the tool dimension, domain dimension, and level dimension three classification tasks, the model achieved F1 values of 95.2%, 93.8%, and 97.1%, respectively, comprehensively surpassing existing methods.

[0153] On the public text element detection task, the LPLC2-BERT model exhibited excellent performance. For the four types of key elements: public text target, implementing subject, supporting measures, and constraint conditions, the model achieved an average precision of 90.7%, an average recall of 88.9%, an average F1 value of 89.8%, and an average detection accuracy (mAP@0.5) of 87.8%. Among them, the implementing subject detection performance was the highest (F1 value 93.3%), and the constraint condition detection performance was relatively low (F1 value 86.4%), but still maintained at a high level.

[0154] In terms of recommendation system evaluation, compared with traditional recommendation methods, the multi-objective optimization recommendation algorithm achieved significant improvement. Compared with content-based recommendation, the click rate increased by 39.6% (from 8.3% to 12.7%); compared with collaborative filtering, the precision increased by 15.7%, and the recall increased by 14.0%. In terms of diversity indicators, this method performed more outstandingly, with coverage rate increasing by 39.8% and novelty increasing by 42.4%. The user satisfaction score increased from 3.8 out of 5 to 4.3 out of 5, an increase of 13.2%.

[0155] In addition, case analysis was conducted to demonstrate the application effect of the model in actual public texts. For example, for a public text on "Special Fund Management Measures for Artificial Intelligence Industry Innovation Development", the model not only correctly classified it as "funding support type / artificial intelligence field / provincial level", but also accurately located the key elements in the text, including the public text target ("promote the deep integration of artificial intelligence and the real economy, and accelerate the innovation and application of artificial intelligence", confidence 95.3%), implementing subject ("provincial science and technology department, provincial finance department", confidence 97.1%), supporting measures ("up to 500,000 yuan of funding for artificial intelligence core technology research and development projects", "up to 300,000 yuan of funding for innovation platform construction", confidence 93.4% and 92.1%), and constraint conditions ("the applicant must be a registered enterprise with more than 2 years of history", "the project must have independent intellectual property rights", confidence 90.2% and 89.7%).

[0156] Such precise element detection capability enables users to quickly locate the most concerned content in public text, greatly improving information acquisition efficiency.

[0157] As shown in Figure 2 The application also provides a deep learning-based intelligent classification and service device for public science and technology text, comprising:

[0158] A data acquisition and preprocessing module 701 is configured to acquire multi-source public science and technology text data, perform text cleaning, deduplication and format unification processing on the multi-source public science and technology text data, and obtain a preprocessed multi-source public science and technology text dataset.

[0159] A feature extraction and enhancement module 702 is configured to extract keywords based on the preprocessed multi-source public science and technology text dataset, and splice the keywords with public text titles to form enhanced text features.

[0160] A classification system construction module 703 is configured to construct a multi-dimensional public text classification system including talent support, fund support, technology support and regulation control based on the enhanced text features, perform data labeling, and establish a training dataset.

[0161] A model training module 704 is configured to perform feature extraction and domain adaptability fine-tuning on the training dataset using a BERT pre-training model, and obtain a public text classification model through parameter optimization and ensemble learning.

[0162] An intelligent analysis and visualization module 705 is configured to automatically classify newly added multi-source public science and technology text data based on the public text classification model to obtain a classification result, analyze public text distribution characteristics using data mining technology, and generate an analysis result chart through a visualization tool.

[0163] A recommendation service module 706 is configured to construct a user portrait based on user interest data including user registration information, browsing history and search keyword data, construct a public text feature index based on the classification result and the enhanced text features, and calculate the similarity between user interest and public text features using a recommendation algorithm based on the user portrait and the public text feature index to generate a personalized public text recommendation result.

[0164] The above description is only preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A deep learning-based intelligent classification and service method for scientific and technological public texts, characterized by: include: Acquire multi-source science and technology public text data, perform text cleaning, deduplication and format unification processing on the multi-source science and technology public text data, and obtain a pre-processed multi-source science and technology public text data set; Based on the preprocessed multi-source science and technology public text dataset, a keyword extraction algorithm is used to extract keywords and splice them with public text titles to form enhanced text features; Based on the enhanced text features, a multi-dimensional public text classification system including talent support, financial support, technical support, and regulatory control is constructed, and data annotation is performed to establish a training data set; The BERT pre-trained model is used to perform feature extraction and domain adaptability fine-tuning on the training dataset, and a common text classification model is obtained through parameter optimization and ensemble learning; Automatically classify the newly added multi-source scientific and technological public text data to be classified based on the public text classification model to obtain classification results; and use data mining technology to analyze the distribution characteristics of public texts, and generate analysis result charts through visualization tools; Based on user interest data including user registration information, browsing history, and search keyword data, a user profile is constructed; and based on the classification results and the enhanced text features, a common text feature index is constructed; and based on the user profile and the common text feature index, a recommendation algorithm is used to calculate the similarity between user interests and common text features to generate personalized common text recommendation results; The method further comprises: When using the BERT pre-trained model for feature extraction and domain adaptation fine-tuning, the LPLC2 neural network is integrated to achieve multi-target detection of public text content, including: Divide the preprocessed multi-source technology public text dataset into text blocks according to sentences and paragraphs, convert the text blocks into high-dimensional vectors using BERT word embedding technology, map them to two-dimensional space through a dimensionality reduction algorithm to retain semantic similarity, construct a two-dimensional semantic feature map, and realize two-dimensional representation conversion of public text; Define key element types including public text objectives, implementation entities, support measures, and constraints; design multi-element annotation specifications; develop auxiliary annotation tools to support the annotation of text areas; and establish a fine-grained public text multi-element annotation system; The BERT model is used to extract the contextual semantic representation of public text. The BERT hidden layer features are converted into two-dimensional feature maps as the input of the LPLC2 network. The local pyramid and contrast modules of LPLC2 are applied to enhance the recognition of key element boundaries. A multi-task learning head is designed to simultaneously output the overall classification and element detection results, and the LPLC2-BERT fusion public text classification model is obtained.

2. The deep learning-based intelligent classification and service method for scientific and technological public texts according to claim 1 is characterized in that: The multi-source science and technology public text data is cleaned, deduplicated, and formatted uniformly to obtain a pre-processed multi-source science and technology public text data set, including: Utilize an automated data collection framework to collect data from websites at all levels, call API interfaces to obtain structured public text information, and form an original multi-source science and technology public text dataset containing title, document number, release date, and text content; The original multi-source science and technology public text dataset is subjected to HTML tag removal and format normalization processing, and duplicate public texts are removed by a hash algorithm to obtain the preprocessed multi-source science and technology public text dataset.

3. The method for intelligent classification and service of scientific and technological public texts based on deep learning according to claim 1 is characterized in that: It also includes structural analysis of the preprocessed multi-source science and technology public text dataset: Based on the characteristics of public text formats, the preprocessed multi-source science and technology public text dataset is decomposed into title, text, and attachment parts, and key information including public text goals, specific measures, and implementation time are extracted from the text to form structured multi-source science and technology public text data; The structured multi-source scientific and technological public text data is subjected to terminology normalization and format unification processing to obtain a standardized multi-source scientific and technological public text data set.

4. The method for intelligent classification and service of scientific and technological public texts based on deep learning according to claim 2 is characterized in that: The keyword extraction algorithm is used to extract keywords and combine them with the public text title to form enhanced text features, including: Obtain a science and technology public text domain dictionary, and use a Chinese word segmentation tool in combination with the science and technology public text domain dictionary to perform word segmentation processing on the preprocessed multi-source science and technology public text dataset to obtain a public text word segmentation result set; Extracting a set of keywords representing the core content of the text from the public text word segmentation result set based on the TextRank algorithm; The keyword set is concatenated with the title of the preprocessed multi-source science and technology public text dataset, and converted into a vector representation using word embedding technology to obtain the enhanced text feature.

5. The method for intelligent classification and service of scientific and technological public texts based on deep learning according to claim 1 is characterized in that: The construction of a multi-dimensional public text classification system including talent support, financial support, technical support, and regulatory control, as well as data annotation and the establishment of a training data set, includes: Design the multi-dimensional public text classification system including the public text tool dimension, the public text field dimension, and the public text level dimension, and formulate classification standards and judgment rules; A stratified sampling method is adopted to select public text samples, and domain experts are organized to manually annotate according to the classification standard and the judgment rule. The annotation quality is ensured through cross-annotation by multiple people to obtain the training data set.

6. The method for intelligent classification and service of scientific and technological public texts based on deep learning according to claim 1 is characterized in that: The training dataset is subjected to feature extraction and domain adaptability fine-tuning using the BERT pre-training model, and a common text classification model is obtained through parameter optimization and ensemble learning, including: Dividing the training data set into a training set and a test set, performing BERT word segmentation processing on the text data of the training data set, and expanding the training samples through data augmentation operations; Fine-tune the Chinese BERT model for domain adaptability, setting parameters including learning rate and batch size, and using the cross-entropy loss function and Adam optimizer for model training; An ensemble learning method is used to combine the prediction results of multiple models to improve classification accuracy and obtain the common text classification model.

7. The method for intelligent classification and service of scientific and technological public texts based on deep learning according to claim 1 is characterized in that: The BERT pre-trained model is used for feature extraction and domain adaptation fine-tuning, including: Based on the preprocessed multi-source science and technology public text dataset, a multi-dimensional feature space including semantic relevance, time relevance, and institutional relevance is constructed, and semantic vectors are calculated and the influence of public texts is evaluated using a word embedding method; An incremental learning strategy is adopted to enable the public text classification model to continuously learn new public text information, an anomaly detection algorithm is used to identify emerging concepts to dynamically expand the feature space, and a public text knowledge graph is established to achieve automatic expansion of concept boundaries; Multiple optimization objectives including classification accuracy, category balance, and diversity are defined, and a multi-objective optimization algorithm is used to find the balance point of each objective to obtain a public text classification model with multi-objective optimization.

8. The method for intelligent classification and service of scientific and technological public texts based on deep learning according to claim 7 is characterized in that: The multi-objective optimization further includes: A diversity balance strategy is introduced to ensure that the personalized public text recommendation results cover different types of public text information, and a diversity reward mechanism is designed to encourage the multi-objective optimized public text classification model to explore public text areas that users have not been exposed to; An attention mechanism is used to dynamically evaluate the importance weight of each feature in the multi-dimensional feature space, and a feature feedback loop is established to adjust the feature weight system through user interaction data; Expand traditional evaluation indicators and introduce diversity indicators including coverage and novelty to build a diversity-aware evaluation system.

9. The method for intelligent classification and service of scientific and technological public texts based on deep learning according to claim 1 is characterized in that: The acquisition of multi-source scientific and technological public text data includes ensuring data collection stability through heterogeneous resource allocation, including: Develop a data collection engine that supports multiple protocols and is compatible with network protocols including HTTP, HTTPS, and FTP. Implement differentiated access control strategies to dynamically adjust collection parameters based on the characteristics of different data sources, build a distributed proxy pool, and obtain stable multi-source scientific and technological public text data collection capabilities. A microservices architecture is established based on container technology. By real-time monitoring and collecting resource usage, a priority task scheduling algorithm is designed to ensure that key data collection tasks are prioritized, thus achieving resource-aware dynamic scheduling. Build a dependency graph for data collection services, apply graph optimization algorithms to calculate the optimal service deployment method, establish edge nodes for data preprocessing, and obtain an optimized multi-source scientific and technological public text data collection architecture.

10. The method for intelligent classification and service of scientific and technological public texts based on deep learning according to claim 9 is characterized in that: The heterogeneous resource allocation also includes quality of service assurance for data transmission and processing, including: The design includes multi-level service quality definitions, including high-priority real-time collection, standard collection, and background batch processing, enabling dynamic perception of network conditions. Adaptive data compression and transmission strategies are adopted according to different QoS levels to ensure the stability and integrity of data collection. Establish a terminal device feature database, develop an adaptive data format conversion engine, implement data processing optimization based on network conditions, and design a progressive data loading strategy to ensure data acquisition efficiency in different network environments; Deploy a distributed data collection and monitoring system, design a data collection performance monitoring panel, implement an intelligent alarm mechanism to automatically trigger the repair process when the data collection quality falls below the threshold, and build an end-to-end data collection quality assurance system.

11. The method for intelligent classification and service of scientific and technological public texts based on deep learning according to claim 1 is characterized in that: The LPLC2-BERT fusion public text classification model also includes attention mechanism enhancement and joint optimization, including: A self-attention mechanism is introduced in the BERT encoding layer to capture long-range dependencies between common text words. A cross-modal attention module is designed to enable the LPLC2 network to focus on key areas of the feature map based on semantic importance. This combines channel attention and spatial attention to enhance feature representation. A hierarchical attention fusion mechanism is constructed to integrate features at different levels to achieve attention-enhanced feature extraction. Construct a joint loss function that includes a common text classification loss and an element detection loss, introduce the consistency loss between BERT representation and LPLC2 features, and design a dynamic weight balancing strategy to adjust the contribution ratio of each loss to achieve multi-task joint optimization training; Implement model quantization and pruning to reduce inference time, develop a non-maximum suppression algorithm to filter redundant detection results, construct a logical association relationship diagram between detection elements, and obtain a structured public text understanding representation and an optimized public text classification model.

12. A deep learning-based intelligent classification and service device for scientific and technological public texts, characterized in that: include: A data collection and preprocessing module is used to obtain multi-source scientific and technological public text data, and perform text cleaning, deduplication and format unification processing on the multi-source scientific and technological public text data to obtain a preprocessed multi-source scientific and technological public text data set; A feature extraction enhancement module is used to extract keywords based on the preprocessed multi-source science and technology public text dataset using a keyword extraction algorithm and combine them with the public text titles to form enhanced text features; A classification system construction module is used to build a multi-dimensional public text classification system including talent support, financial support, technical support, and legal control based on the enhanced text features, and to perform data annotation and establish a training data set; A model training module is used to perform feature extraction and domain adaptability fine-tuning on the training dataset using the BERT pre-trained model, and obtain a common text classification model through parameter optimization and ensemble learning; An intelligent analysis and visualization module is used to automatically classify the newly added multi-source scientific and technological public text data to be classified based on the public text classification model to obtain classification results; and use data mining technology to analyze the distribution characteristics of public texts and generate analysis result charts through visualization tools; A recommendation service module is configured to construct a user profile based on user interest data including user registration information, browsing history, and search keyword data; construct a common text feature index based on the classification results and the enhanced text features; and, based on the user profile and the common text feature index, use a recommendation algorithm to calculate the similarity between user interests and common text features to generate personalized common text recommendation results; It also includes a multi-target detection module for fusing the LPLC2 neural network to achieve multi-target detection of public text content when using the BERT pre-trained model for feature extraction and domain adaptation fine-tuning, including: Divide the preprocessed multi-source technology public text dataset into text blocks according to sentences and paragraphs, convert the text blocks into high-dimensional vectors using BERT word embedding technology, map them to two-dimensional space through a dimensionality reduction algorithm to retain semantic similarity, construct a two-dimensional semantic feature map, and realize two-dimensional representation conversion of public text; Define key element types including public text objectives, implementation entities, support measures, and constraints; design multi-element annotation specifications; develop auxiliary annotation tools to support the annotation of text areas; and establish a fine-grained public text multi-element annotation system; The BERT model is used to extract the contextual semantic representation of public text. The BERT hidden layer features are converted into two-dimensional feature maps as the input of the LPLC2 network. The local pyramid and contrast modules of LPLC2 are applied to enhance the recognition of key element boundaries. A multi-task learning head is designed to simultaneously output the overall classification and element detection results, and the LPLC2-BERT fusion public text classification model is obtained.

Images