An asphalt pavement technical document mining method based on TongYiQianWan semantic graph

Abstract

The application discloses a kind of asphalt pavement technical literature mining methods based on TongYiQianWan semantic graph, comprising: obtaining road engineering-computer vision cross field academic literature data, carries out multidimensional preprocessing and forms standardized data set;Improved double-tower semantic coding network is constructed, the uniform mapping of keyword and title semantic features is realized;Optimized GraphSAGE spatial feature extraction network is constructed;CNN-LSTM hybrid architecture is used to construct timing feature extraction network, and form a multidimensional fusion feature representation system;Weighted random forest classifier with class weight balance strategy is introduced;Key word co-occurrence prediction and technology trend quantitative prediction system is established, finally, multi-module collaborative Qwen-SGCL intelligent model is obtained;The training, verification and comprehensive performance evaluation of the model are completed.The application provides an efficient intelligent tool for intelligent mining and technology trend quantitative analysis of literature in the cross field of road engineering and computer vision.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of road engineering and artificial intelligence, and in particular to a method for mining technical documents on asphalt pavement based on a semantic graph of a thousand questions. Background Technology

[0002] The deep integration of road engineering and computer vision is a core direction for the development of intelligent transportation and smart construction, and its technological innovations are concentrated in academic literature. Efficiently mining literature resources in this interdisciplinary field, accurately identifying cross-disciplinary technological connections, and scientifically predicting technological development trends are of significant engineering value and academic importance for guiding the technological research and development direction in the field of road engineering, optimizing the allocation of scientific research resources, and accelerating the transformation of research results.

[0003] Currently, cross-disciplinary literature mining mainly relies on traditional keyword matching, single machine learning classification, and general large-scale model retrieval techniques, which suffer from three major drawbacks: First, the accuracy of cross-disciplinary technology association identification is low; second, it is difficult to quantitatively predict technological trends, as existing methods are mostly based on bibliometrics for qualitative analysis, failing to quantitatively characterize technological evolution paths and accurately predict future development trends; and third, the ability to fuse multi-dimensional features is insufficient, failing to effectively integrate the semantic features of documents, the spatial structural features of technological associations, and the temporal features of technological evolution, resulting in poor model generalization ability and robustness. To address these shortcomings, this invention proposes a multi-module collaborative Qwen-SGCL (Qwen-based Semantic Graph Collaborative Learning Model) intelligent model, providing an efficient intelligent tool for intelligent literature mining and quantitative analysis of technological trends in the cross-disciplinary fields of road engineering and computer vision. Summary of the Invention

[0004] Purpose of the invention: To address the shortcomings of the existing technologies, this invention proposes a method for mining technical documents on asphalt pavement based on a semantic graph of a thousand questions.

[0005] The technical solution adopted in this invention is: a method for mining asphalt pavement technical literature based on the semantic graph of a thousand questions, comprising the following steps:

[0006] S1: Acquire academic literature data in the interdisciplinary field of road engineering and computer vision, and perform multi-dimensional preprocessing to form a standardized dataset;

[0007] S2: Construct an improved dual-tower semantic coding network, using the Qwen-1.8B-Chat large model as a shared semantic base, and introduce non-linear activation functions and Dropout regularization to achieve a unified mapping between keywords and title semantic features;

[0008] S3: Construct an optimized GraphSAGE spatial feature extraction network, add learnable bias vectors to enhance the ability to extract spatial structural features related to cross-domain technologies; construct a temporal feature extraction network using a CNN-LSTM hybrid architecture, integrate semantic-spatial-temporal three-dimensional features to form a multi-dimensional fusion feature representation system; and adopt a weighted random forest classifier that introduces a class weight balancing strategy.

[0009] S4: Establish a keyword co-occurrence prediction and technology trend quantitative prediction system, and finally obtain the Qwen-SGCL intelligent model based on multi-module collaboration;

[0010] S5: Complete the training, validation, and comprehensive performance evaluation of the model, and calculate and compare the various evaluation metrics of the Qwen-SGCL model with each comparison model, including: AUC, precision, recall, F1 score, and AUPR.

[0011] Furthermore, the S1 process is implemented through the following steps: multi-dimensional preprocessing of the acquired original literature data, including: literature metadata cleaning (removing duplicate literature and completing missing fields), text standardization (unifying Chinese terminology and removing special characters and stop words); and constructing a standardized dataset containing literature titles, keywords, and publication years.

[0012] Furthermore, the S2 process is implemented through the following steps: extracting global semantic features of the text through a unified Qwen-1.8B-Chat large model base, designing independent Siamese fully connected branches for keyword nodes and document title edges, completing the mapping of high-dimensional semantic features to task-adaptive low-dimensional dense features, and finally eliminating the influence of dimensions through normalization to output standardized text semantic features that can be directly fused with spatial and temporal features.

[0013] The module first performs word segmentation, padding, and masking on the input unstructured text (keywords / titles), converting the text into a numerical tensor that can be processed by a large language model. Mathematically, this is expressed as:

[0014] For batch text lists ( (where the batch size is 1), after being encoded by a tokenizer, a sequence of input indices of varying lengths is obtained. ( For the first (Text length), uniformly set to a fixed length. :

[0015] (1)

[0016] In the formula, This represents the output sequence after length normalization. For the first The original input sequence; For the first The original length of the sequence; The preset maximum sequence length threshold; This represents a sequence concatenation operation. Indicates length is The filling vector, These are preset fill tags used to mark invalid words;

[0017] Finally, an equal-length input tensor is generated. ;

[0018] (2)

[0019] In the formula, For attention mask tensors; For the first The original length of the sequence;

[0020] The encoded tensor is input into the frozen / semi-frozen Qwen-1.8B-Chat large model to extract the global high-dimensional semantic features of the text. The mathematical operations corresponding to the semantic core of this module are as follows:

[0021] Model forward propagation: and Input the Transformer encoder of Qwen-1.8B-Chat, via Layer-wise self-attention and feedforward network computation, outputting the hidden states of each layer. , of which Hidden state of the last layer It contains the most abstract textual semantic information;

[0022] The feature at position 0 in the last hidden state is selected as the global semantic representation of a single text, using the following formula:

[0023] (3)

[0024] In the formula, These are the basic semantic features of Qwen; The hidden state tensor of the last Transformer layer in Qwen-1.8B-Chat ( ); This represents the fixed hidden dimension of Qwen-1.8B-Chat;

[0025] To address the differences in textual characteristics between keyword nodes and document title edges, a twin-tower network with independent structure and non-shared weights is designed. The 2048-dimensional high-dimensional semantic features are mapped to low-dimensional dense features of the dimension specified by the code. At the same time, non-linear activation and Dropout regularization are introduced to improve feature expressiveness and prevent overfitting. The mathematical principles of the two branches are consistent.

[0026] Keyword node branch (mapped to 16-dimensional node features), this branch completes Feature dimensionality reduction mapping; document title side branch (mapping to 256-dimensional side features), this branch completes Feature dimensionality reduction mapping; the formula is:

[0027] (4)

[0028] In the formula, The original semantic features are the input. Batch size; The intermediate hidden dimension of the first fully connected layer of the node branch; The intermediate hidden dimension of the first fully connected layer of the edge branch; The intermediate hidden layer features are the output of the first layer operation of the node branch; These are the intermediate features after the first layer activation of the edge branch; , The weights and biases of the first fully connected layer; , The weights and biases of the first fully connected layer in the title branch; ReLU is the non-linear activation function;

[0029] (5)

[0030] In the formula, The intermediate hidden layer features are the output of the first layer operation of the node branch; These are the intermediate features after the first layer activation of the edge branch; Intermediate features after Dropout processing of node branches; These are intermediate features after Dropout processing of the edge branches; For random inactivation regularization; Batch size; The intermediate hidden dimension of the first fully connected layer of the node branch; The intermediate hidden dimension of the first fully connected layer of the edge branch;

[0031] (6)

[0032] In the formula, Batch size; The hidden dimension of the fully connected nodes; The hidden dimension of the fully connected edge branch; Intermediate features after Dropout processing of node branches; These are intermediate features after Dropout processing of the edge branches; Features of the final result of node branches; Features of the final result of the edge branch; , The weights and biases for the second fully connected layer; , The weights and biases of the second fully connected layer in the title branch;

[0033] Dropout is a random deactivation regularization, with The probability is randomly set to zero for some feature values ​​to avoid branch overfitting;

[0034] The activation function, regularization strategy, and node branches are consistent to ensure the consistency of feature transformation.

[0035] Furthermore, the S3 process is implemented through the following steps: taking text node features as input, a multi-layer mean aggregation GraphSAGE network is constructed to capture the co-occurrence associations between keywords and the spatial structure features of the knowledge graph; the network input is 16-dimensional text node features, which are enhanced by GraphSAGE convolutional layers, and finally output 256-dimensional spatial features. Each intermediate layer introduces ReLU activation function, Dropout regularization and batch normalization.

[0036] No. The node feature update formula for layer GraphSAGE is as follows:

[0037] (7)

[0038] In the formula, It is a connection node and The weight of the edge; It is a node In the The feature vector of the layer; It is a node The neighbor aggregated feature vector;

[0039] (8)

[0040] In the formula, It is a node The neighbor aggregated feature vector; The target node In the The feature vector of the layer; It is a vector concatenation operation; It is the first Learnable weight matrix for layer graph convolution; It is the first Learnable bias vectors for layer graph convolution; It is the intermediate feature vector after linear transformation;

[0041] (9)

[0042] In the formula, It is the intermediate feature vector after linear transformation; It is a non-linear activation function, specifically ReLU; It is a node In the The output feature vector after layer graph convolution; This represents random deactivation regularization with a dropout rate of 0.5; Indicates batch normalization;

[0043] (10)

[0044] In the formula, It is the final output feature vector of the entire GraphSAGE module; It is the first Layer node feature vectors; Represents the L2 norm;

[0045] The CNN-LSTM temporal evolution feature extraction module first concatenates the spatial features of each year's nodes in chronological order to form a temporal feature sequence of the nodes. It then extracts local spatiotemporal correlation features of the sequence using a one-dimensional CNN, employs Same padding to ensure feature dimension consistency, and adjusts the feature sequence dimensions to a format suitable for the LSTM network after batch normalization and ReLU activation. Next, a unidirectional LSTM network captures the long-term temporal dependencies of the sequence, extracting the core features of temporal evolution. Finally, these features are mapped to 256-dimensional temporal features through a fully connected layer and L2 normalized.

[0046] The fully connected layer transformation maps the LSTM output to the target temporal feature dimension, as shown in the formula:

[0047] (11)

[0048] In the formula, This is the final output temporal feature tensor; For random inactivation regularization; This represents the probability of dropping data in Dropout. The output features of the LSTM at the last time step; For fully connected weights; For bias; For output dimensions; Batch size;

[0049] Used for CNN-LSTM training, fitting temporal features to spatial features of the last year, formula:

[0050] (12)

[0051] In the formula, This represents the mean squared error loss value; Batch size; This represents the total number of feature dimensions. The predicted temporal features output by the CNN-LSTM network, the first... The node, the first Dimensional features; The real spatial features obtained by encoding in the last year, the first The node, the first Dimensional features;

[0052] We assign weights of 3.6 and 2.95 to the text features of keyword nodes and the text features of title edges, respectively. Then, we concatenate the weighted text features with spatial and temporal features to obtain the node's fusion features. For each pair of keywords, we extract the fusion features of the two nodes and concatenate them again with the fusion features of the corresponding edges to form the final keyword pair association features. The formula is:

[0053] (13)

[0054] In the formula, Features of the original graph nodes; These are the edge features of the original graph; The scaled graph node features; The scaled-down graph edge features; , Set a fixed scaling factor for the feature;

[0055] (14)

[0056] In the formula, The scaled graph node features; Static attribute features of nodes; For node temporal dynamic characteristics; For node-level multimodal fusion features, among which ; For a set of keywords, the node fusion features of each keyword; , , These correspond to the dimensions of nodes, static features, and temporal features, respectively. Batch size;

[0057] (15)

[0058] In the formula, The scaled-down graph edge features; These correspond to the dimensions of nodes, static features, and temporal features, respectively. This represents the final total dimension of the node fusion features; For the final multi-source fusion features, among which ; This represents the total dimension of the final features; Batch size; The hidden dimension of the fully connected edge branch;

[0059] Using multi-dimensional fused features as input, a weighted random forest classifier is constructed to predict keyword associations; all CPU cores are utilized to improve training and inference efficiency; the decision tree splitting of the random forest is based on the Gini coefficient, formula:

[0060] (16)

[0061] In the formula, It is a sample set The Middle The proportion of positive / negative samples; For Gini impurity.

[0062] Furthermore, the S4 process is implemented through the following steps: a keyword co-occurrence prediction and technology trend prediction system for the academic field is constructed; the lightweight prediction inference module first performs word matching verification based on the checkpoints of the pre-trained model for the paired keywords input by the user, and directly reuses the pre-calculated feature vectors for the known words already included in the corpus; for the cutting-edge new words and unfamiliar cross-domain words not included, the zero-shot inference mode is started, and the semantic feature extraction is completed in real time through the pre-trained large language model, and finally the real-time calculation of the probability of pairwise keyword association is realized.

[0063] The technology trend prediction module is based on the annual word frequency statistics of literature keywords and uses the Prophet model to conduct trend prediction. It sets a prior scale of 0.05 for trend change points to balance the model's fitting ability and overfitting risk, taking into account the characteristics of the annual literature data. Based on historical data, it extrapolates trends for the next five years. It can quantitatively identify emerging hot topics in their growth phase, declining trends, and stable basic research areas. By associating core keywords within the field and consulting with relevant technical personnel in the civil engineering field, it reasonably infers the development direction of technology and the related fields that the technology may involve. Finally, it obtains a multi-module collaborative Qwen-SGCL intelligent model.

[0064] Furthermore, the S5 process is implemented through the following steps: To verify the effectiveness of the Qwen-SGCL model, representative baseline models are selected as comparison objects; these include: firstly, traditional machine learning algorithms, including logistic regression (LogR), support vector machine (SVM), random forest (RF), XGBoost, and AdaBoost; and secondly, current mainstream advanced graph learning models, including graph attention networks (GAT) and graph convolutional networks (GCN); 10-fold hierarchical cross-validation is used to ensure the robustness of the results, and the experiment is repeated and the average value is taken; the evaluation metrics include:

[0065] (1) Area under the ROC curve (AUC):

[0066] (17)

[0067] (18)

[0068] In the formula, For the first False positive rate per sample; For the first The true positive rate per sample; TP is the number of samples correctly predicted as positive by the model; FP is the number of samples incorrectly predicted as positive by the model; TN is the number of samples correctly predicted as negative by the model; FN is the number of samples incorrectly predicted as negative by the model.

[0069] (2) Precision:

[0070] (19)

[0071] In the formula, TP represents the number of samples correctly predicted as positive by the model; FP represents the number of samples incorrectly predicted as positive by the model.

[0072] (3) Recall:

[0073] (20)

[0074] In the formula, TP represents the number of samples correctly predicted as positive by the model; FN represents the number of samples incorrectly predicted as negative by the model.

[0075] (4) Harmonic mean of precision and recall (F1 Score):

[0076] (twenty one)

[0077] (5) Area under the PR curve (AUPR):

[0078] (twenty two)

[0079] In the formula, , For the first , No. Recall rate corresponding to each threshold; , For the first , No. The accuracy corresponding to each threshold.

[0080] The beneficial effects of the present invention are: (1) as follows Figure 2 As shown, the Qwen-SGCL model achieved the best results across all models in terms of AUC, AUPR, and F1 score, reaching 0.8779, 0.93, and 0.8445, respectively. Specifically, its AUC improved by 1.53 percentage points compared to the second-best XGBoost model, its AUPR improved by 2.69 percentage points compared to the second-best LogR model, and its F1 score improved by 3.9 percentage points compared to the second-best LogR model, demonstrating a significant advantage in core overall performance. Furthermore, the Qwen-SGCL model achieved a precision of 0.8619, second only to RF and AdaBoost; and a recall of 0.8278, second only to LogR and SVM. Compared to RF and AdaBoost, which have high precision but low recall (less than 0.4), Qwen-SGCL improves recall by 54.47 and 44.68 percentage points, respectively. Compared to LogR and SVM, which have high recall but low precision (less than 0.7), Qwen-SGCL improves precision by 16.47 and 25.49 percentage points, respectively, addressing the imbalance issue of traditional baseline models. Furthermore, in terms of AUPR, Qwen-SGCL is the only model to break through 0.93, improving upon LogR (the best-performing traditional machine learning model) by 2.69 percentage points and GCN (the best-performing graph learning model) by 10.57 percentage points, fully validating the model's strong generalization ability and robustness on imbalanced datasets.

[0081] (2) A system for predicting the co-occurrence of keywords in academic fields and forecasting technological trends was constructed. A dual-track lightweight inference module with pre-computation feature reuse and zero-shot real-time inference was designed, solving the pain point of high inference cost in traditional models. For example... Figure 3 As shown, the embedded Prophet time series model predicts the frequency of occurrence of the top 10 trending keywords over the next 5 years. The prediction results can provide data-driven decision support for engineering professionals to grasp the industrialization direction of new technologies in the field.

[0082] in accordance with Figure 3 The frequency differences in keywords clearly categorize keyword development trends into three types: rising, stable, and declining. Table 1 shows the future development trends of core keywords in the field. By associating core keywords with relevant technical and production personnel in the civil engineering field, reasonable predictions can be made regarding the direction of technological development and the related fields that the technology may involve. Table 2 shows the main situation of the three major future technological fields.

[0083] Table 1. Future Development Trends of Core Keywords in the Field

[0084]

[0085] Table 2 Examples of Core Keywords in the Field and Potential Technologies

[0086] Attached Figure Description

[0087] Figure 1 This is a diagram of the model architecture of the present invention.

[0088] Figure 2 This is a schematic diagram comparing the performance of the models in this invention.

[0089] Figure 3 This is a schematic diagram showing the frequency and prediction of the top 10 core keywords in the field of this invention. Detailed Implementation

[0090] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0091] This invention uses academic literature in the interdisciplinary field of road engineering and computer vision, indexed in databases such as CNKI, from 2016 to 2025, as its research object. Search keywords include "asphalt mixture" and "image processing." The literature data includes core information such as keywords, paper titles, and publication years. A research sample of 778 Chinese papers was constructed.

[0092] The experiment was implemented using the PyTorch 2.1.0 framework and conducted in a hybrid configuration strategy prioritizing GPU performance while maintaining CPU compatibility. NumPy version 1.26.4 was also adapted to ensure environmental compatibility. The parameter configurations were as follows: batch size set to 16; keyword node feature dimension set to 16; document title edge feature dimension set to 256; learning rate set to 0.001, at which point the model's convergence speed and stability reached their optimal levels. The GraphSAGE spatial feature extraction module used a 3-layer network structure with a hidden layer dimension of 256 and a neighbor sampling number of 15. The CNN-LSTM temporal feature module used a 3-layer LSTM with a hidden dimension of 512, a CNN kernel size of 3, and 512 output channels. To adapt to the imbalanced data features of technology fusion relationship prediction, the number of decision trees was set to 350, the maximum depth to 24, and the minimum number of split samples to 2. Positive samples were assigned a weight of 2, and negative samples a weight of 1. The feature selection strategy adopted the log2 criterion to limit the number of features and reduce the risk of overfitting.

[0093] like Figure 1 As shown, a method for mining asphalt pavement technical literature based on the semantic graph of a thousand questions includes the following steps:

[0094] S1: Acquire academic literature data in the interdisciplinary field of road engineering and computer vision, and perform multi-dimensional preprocessing to form a standardized dataset;

[0095] The S1 process is implemented through the following steps: multi-dimensional preprocessing of the acquired original literature data, including: literature metadata cleaning (removing duplicate literature and completing missing fields), text standardization (unifying Chinese terminology and removing special characters and stop words); and constructing a standardized dataset containing literature titles, keywords, and publication years.

[0096] S2: Construct an improved dual-tower semantic coding network, using the Qwen-1.8B-Chat large model as a shared semantic base, and introduce non-linear activation functions and Dropout regularization to achieve a unified mapping between keywords and title semantic features;

[0097] The S2 process is implemented through the following steps: extracting global semantic features of the text through a unified Qwen-1.8B-Chat large model base, designing independent Siamese fully connected branches for keyword nodes and document title edges, completing the mapping of high-dimensional semantic features to task-adaptive low-dimensional dense features, and finally eliminating the influence of dimensions through normalization to output standardized text semantic features that can be directly fused with spatial and temporal features.

[0098] The module first performs word segmentation, padding, and masking on the input unstructured text (keywords / titles), converting the text into a numerical tensor that can be processed by a large language model. Mathematically, this is expressed as:

[0099] For batch text lists ( (where the batch size is 1), after being encoded by a tokenizer, a sequence of input indices of varying lengths is obtained. ( For the first (Text length), uniformly set to a fixed length. :

[0100] (1)

[0101] In the formula, This represents the output sequence after length normalization. For the first The original input sequence; For the first The original length of the sequence; The preset maximum sequence length threshold; This represents a sequence concatenation operation. Indicates length is The filling vector, These are preset fill tags used to mark invalid words;

[0102] Finally, an equal-length input tensor is generated. ;

[0103] (2)

[0104] In the formula, For attention mask tensors; For the first The original length of the sequence;

[0105] The encoded tensor is input into the frozen / semi-frozen Qwen-1.8B-Chat large model to extract the global high-dimensional semantic features of the text. The mathematical operations corresponding to the semantic core of this module are as follows:

[0106] Model forward propagation: and Input the Transformer encoder of Qwen-1.8B-Chat, via Layer-wise self-attention and feedforward network computation, outputting the hidden states of each layer. , of which Hidden state of the last layer It contains the most abstract textual semantic information;

[0107] The feature at position 0 in the last hidden state is selected as the global semantic representation of a single text, using the following formula:

[0108] (3)

[0109] In the formula, These are the basic semantic features of Qwen; The hidden state tensor of the last Transformer layer in Qwen-1.8B-Chat ( ); This represents the fixed hidden dimension of Qwen-1.8B-Chat;

[0110] To address the differences in textual characteristics between keyword nodes and document title edges, a twin-tower network with independent structure and non-shared weights is designed. The 2048-dimensional high-dimensional semantic features are mapped to low-dimensional dense features of the dimension specified by the code. At the same time, non-linear activation and Dropout regularization are introduced to improve feature expressiveness and prevent overfitting. The mathematical principles of the two branches are consistent.

[0111] Keyword node branch (mapped to 16-dimensional node features), this branch completes Feature dimensionality reduction mapping; document title side branch (mapping to 256-dimensional side features), this branch completes Feature dimensionality reduction mapping; the formula is:

[0112] (4)

[0113] In the formula, The original semantic features are the input. Batch size; The intermediate hidden dimension of the first fully connected layer of the node branch; The intermediate hidden dimension of the first fully connected layer of the edge branch; The intermediate hidden layer features are the output of the first layer operation of the node branch; These are the intermediate features after the first layer activation of the edge branch; , The weights and biases of the first fully connected layer; , The weights and biases of the first fully connected layer in the title branch; ReLU is the non-linear activation function;

[0114] (5)

[0115] In the formula, The intermediate hidden layer features are the output of the first layer operation of the node branch; These are the intermediate features after the first layer activation of the edge branch; Intermediate features after Dropout processing of node branches; These are intermediate features after Dropout processing of the edge branches; For random inactivation regularization; Batch size; The intermediate hidden dimension of the first fully connected layer of the node branch; The intermediate hidden dimension of the first fully connected layer of the edge branch;

[0116] (6)

[0117] In the formula, Batch size; The hidden dimension of the fully connected nodes; The hidden dimension of the fully connected edge branch; Intermediate features after Dropout processing of node branches; These are intermediate features after Dropout processing of the edge branches; Features of the final result of node branches; Features of the final result of the edge branch; , The weights and biases for the second fully connected layer; , The weights and biases of the second fully connected layer in the title branch;

[0118] Dropout is a random deactivation regularization, with The probability is randomly set to zero for some feature values ​​to avoid branch overfitting;

[0119] The activation function, regularization strategy, and node branches are consistent to ensure the consistency of feature transformation.

[0120] S3: Construct an optimized GraphSAGE spatial feature extraction network, add learnable bias vectors to enhance the ability to extract spatial structural features related to cross-domain technologies; construct a temporal feature extraction network using a CNN-LSTM hybrid architecture, integrate semantic-spatial-temporal three-dimensional features to form a multi-dimensional fusion feature representation system; and adopt a weighted random forest classifier that introduces a class weight balancing strategy.

[0121] The S3 process is implemented through the following steps: taking text node features as input, a multi-layer mean aggregation GraphSAGE network is constructed to capture the co-occurrence associations between keywords and the spatial structure features of the knowledge graph; the network input is 16-dimensional text node features, which are enhanced by GraphSAGE convolutional layers, and finally output 256-dimensional spatial features. Each intermediate layer introduces ReLU activation function, Dropout regularization and batch normalization.

[0122] No. The node feature update formula for layer GraphSAGE is as follows:

[0123] (7)

[0124] In the formula, It is a connection node and The weight of the edge; It is a node In the The feature vector of the layer; It is a node The neighbor aggregated feature vector;

[0125] (8)

[0126] In the formula, It is a node The neighbor aggregated feature vector; The target node In the The feature vector of the layer; It is a vector concatenation operation; It is the first Learnable weight matrix for layer graph convolution; It is the first Learnable bias vectors for layer graph convolution; It is the intermediate feature vector after linear transformation;

[0127] (9)

[0128] In the formula, It is the intermediate feature vector after linear transformation; It is a non-linear activation function, specifically ReLU; It is a node In the The output feature vector after layer graph convolution; This represents random deactivation regularization with a dropout rate of 0.5; Indicates batch normalization;

[0129] (10)

[0130] In the formula, It is the final output feature vector of the entire GraphSAGE module; It is the first Layer node feature vectors; Represents the L2 norm;

[0131] The CNN-LSTM temporal evolution feature extraction module first concatenates the spatial features of each year's nodes in chronological order to form a temporal feature sequence of the nodes. It then extracts local spatiotemporal correlation features of the sequence using a one-dimensional CNN, employs Same padding to ensure feature dimension consistency, and adjusts the feature sequence dimensions to a format suitable for the LSTM network after batch normalization and ReLU activation. Next, a unidirectional LSTM network captures the long-term temporal dependencies of the sequence, extracting the core features of temporal evolution. Finally, these features are mapped to 256-dimensional temporal features through a fully connected layer and L2 normalized.

[0132] The fully connected layer transformation maps the LSTM output to the target temporal feature dimension, as shown in the formula:

[0133] (11)

[0134] In the formula, This is the final output temporal feature tensor; For random inactivation regularization; This represents the probability of dropping data in Dropout. The output features of the LSTM at the last time step; For fully connected weights; For bias; For output dimensions; Batch size;

[0135] Used for CNN-LSTM training, fitting temporal features to spatial features of the last year, formula:

[0136] (12)

[0137] In the formula, This represents the mean squared error loss value; Batch size; This represents the total number of feature dimensions. The predicted temporal features output by the CNN-LSTM network, the first... The node, the first Dimensional features; The real spatial features obtained by encoding in the last year, the first The node, the first Dimensional features;

[0138] We assign weights of 3.6 and 2.95 to the text features of keyword nodes and the text features of title edges, respectively. Then, we concatenate the weighted text features with spatial and temporal features to obtain the node's fusion features. For each pair of keywords, we extract the fusion features of the two nodes and concatenate them again with the fusion features of the corresponding edges to form the final keyword pair association features. The formula is:

[0139] (13)

[0140] In the formula, Features of the original graph nodes; These are the edge features of the original graph; The scaled graph node features; The scaled-down graph edge features; , Set a fixed scaling factor for the feature;

[0141] (14)

[0142] In the formula, The scaled graph node features; Static attribute features of nodes; For node temporal dynamic characteristics; For node-level multimodal fusion features, among which ; For a set of keywords, the node fusion features of each keyword; , , These correspond to the dimensions of nodes, static features, and temporal features, respectively. Batch size;

[0143] (15)

[0144] In the formula, The scaled-down graph edge features; These correspond to the dimensions of nodes, static features, and temporal features, respectively. This represents the final total dimension of the node fusion features; For the final multi-source fusion features, among which ; This represents the total dimension of the final features; Batch size; The hidden dimension of the fully connected edge branch;

[0145] Using multi-dimensional fused features as input, a weighted random forest classifier is constructed to predict keyword associations; all CPU cores are utilized to improve training and inference efficiency; the decision tree splitting of the random forest is based on the Gini coefficient, formula:

[0146] (16)

[0147] In the formula, It is a sample set The Middle The proportion of positive / negative samples; For Gini impurity.

[0148] S4: Establish a keyword co-occurrence prediction and technology trend quantitative prediction system, and finally obtain the Qwen-SGCL intelligent model based on multi-module collaboration;

[0149] The S4 process is implemented through the following steps: a keyword co-occurrence prediction and technology trend prediction system for the academic field is constructed; the lightweight prediction inference module first performs word matching verification based on the checkpoints of the pre-trained model for the paired keywords input by the user, and directly reuses the pre-calculated feature vectors for the known words already included in the corpus; for the cutting-edge new words and unfamiliar words across fields that are not included, the zero-shot inference mode is started, and the semantic feature extraction is completed in real time through the pre-trained large language model, and finally the real-time calculation of the probability of pairwise keyword association is realized.

[0150] The technology trend prediction module is based on the annual word frequency statistics of literature keywords and uses the Prophet model to conduct trend prediction. It sets a prior scale of 0.05 for trend change points to balance the model's fitting ability and overfitting risk, taking into account the characteristics of the annual literature data. Based on historical data, it extrapolates trends for the next five years. It can quantitatively identify emerging hot topics in their growth phase, declining trends, and stable basic research areas. By associating core keywords within the field and consulting with relevant technical personnel in the civil engineering field, it reasonably infers the development direction of technology and the related fields that the technology may involve. Finally, it obtains a multi-module collaborative Qwen-SGCL intelligent model.

[0151] S5: Complete the training, validation, and comprehensive performance evaluation of the model, calculate and compare the various evaluation metrics of the Qwen-SGCL model with each comparison model, specifically including: AUC, precision, recall, F1 score, and AUPR;

[0152] The S5 process is implemented through the following steps: To verify the effectiveness of the Qwen-SGCL model, representative baseline models are selected as comparison objects; firstly, traditional machine learning algorithms, including logistic regression (LogR), support vector machine (SVM), random forest (RF), XGBoost, and AdaBoost; secondly, current mainstream advanced graph learning models, including graph attention network (GAT) and graph convolutional network (GCN); 10-fold hierarchical cross-validation is used to ensure the robustness of the results, and the experiment is repeated and the average value is taken; the evaluation metrics include:

[0153] (1) Area under the ROC curve (AUC):

[0154] (17)

[0155] (18)

[0156] In the formula, For the first False positive rate per sample; For the first The true positive rate per sample; TP is the number of samples correctly predicted as positive by the model; FP is the number of samples incorrectly predicted as positive by the model; TN is the number of samples correctly predicted as negative by the model; FN is the number of samples incorrectly predicted as negative by the model.

[0157] (2) Precision:

[0158] (19)

[0159] In the formula, TP represents the number of samples correctly predicted as positive by the model; FP represents the number of samples incorrectly predicted as positive by the model.

[0160] (3) Recall:

[0161] (20)

[0162] In the formula, TP represents the number of samples correctly predicted as positive by the model; FN represents the number of samples incorrectly predicted as negative by the model.

[0163] (4) Harmonic mean of precision and recall (F1 Score):

[0164] (twenty one)

[0165] (5) Area under the PR curve (AUPR):

[0166] (twenty two)

[0167] In the formula, , For the first , No. Recall rate corresponding to each threshold; , For the first , No. The accuracy corresponding to each threshold.

[0168] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for asphalt pavement technical document mining based on TongYiQianWen semantic graph, characterized in that: Includes the following steps: S1: Acquire academic literature data in the interdisciplinary field of road engineering and computer vision, and perform multi-dimensional preprocessing to form a standardized dataset; S2: Construct an improved dual-tower semantic coding network, using the Qwen-1.8B-Chat large model as a shared semantic base, and introduce non-linear activation functions and Dropout regularization to achieve a unified mapping between keywords and title semantic features; S3: Construct an optimized GraphSAGE spatial feature extraction network, add learnable bias vectors to enhance the ability to extract spatial structural features related to cross-domain technologies; use a CNN-LSTM hybrid architecture to construct a temporal feature extraction network, integrate semantic-spatial-temporal three-dimensional features to form a multi-dimensional fusion feature representation system; A weighted random forest classifier with a class weight balancing strategy is adopted; S4: Establish a keyword co-occurrence prediction and technology trend quantitative prediction system, and finally obtain the Qwen-SGCL intelligent model based on multi-module collaboration; S5: Complete the training, validation, and comprehensive performance evaluation of the model, and calculate and compare the various evaluation metrics of the Qwen-SGCL model with each comparison model, including: AUC, precision, recall, F1 score, and AUPR.

2. The method according to claim 1, wherein the method is characterized by: The S1 process is implemented through the following steps: multi-dimensional preprocessing of the acquired original literature data, including: literature metadata cleaning and text standardization; and constructing a standardized dataset containing literature titles, keywords, and publication years.

3. The method according to claim 2, wherein the method is characterized by: The S2 process is implemented through the following steps: extracting global semantic features of the text through a unified Qwen-1.8B-Chat large model base, designing independent Siamese fully connected branches for keyword nodes and document title edges, completing the mapping of high-dimensional semantic features to task-adaptive low-dimensional dense features, and finally eliminating the influence of dimensions through normalization to output standardized text semantic features that can be directly fused with spatial and temporal features. The module first performs word segmentation, padding, and masking on the input unstructured text, converting the text into a numerical tensor that can be processed by a large language model. Mathematically, this is expressed as: For batch text list , Batch size, after Tokenizer encoding, get unequal length input index sequence , The first Text length, uniform fixed length : （1）， In the formula, This represents the output sequence after length normalization. For the first The original input sequence; For the first The original length of the sequence; The preset maximum sequence length threshold; This represents a sequence concatenation operation. Indicates length is The filling vector, These are preset fill tags used to mark invalid words; Finally, an equal-length input tensor is generated. ; （2）， wherein is an attention mask tensor; is the first is the original length of the sequence. The encoded tensor is input into the frozen / semi-frozen Qwen-1.8B-Chat large model to extract the global high-dimensional semantic features of the text. The mathematical operations corresponding to the semantic core of this module are as follows: Model forward pass: pass and input to the Transformer encoder of Qwen-1.8B-Chat, compute layer self-attention and feedforward network, output each layer hidden state where the layer hidden state contains the most abstract text semantic information; The feature at position 0 in the last hidden state is selected as the global semantic representation of a single text, using the following formula: （3）， wherein, is the Qwen base semantic feature; is the Qwen-1.8B-Chat last layer Transformer’s hidden state tensor; denotes the fixed hidden dimension of Qwen-1.8B-Chat; To address the differences in textual characteristics between keyword nodes and document title edges, a twin-tower network with independent structure and non-shared weights is designed. The 2048-dimensional high-dimensional semantic features are mapped to low-dimensional dense features of the dimension specified by the code. At the same time, non-linear activation and Dropout regularization are introduced to improve feature expressiveness and prevent overfitting. The mathematical principles of the two branches are consistent. keyword node branch, which accomplishes dimensionality reduction mapping of features; document title side branch, which accomplishes dimensionality reduction mapping of features; formula is: （4）， In the formula, The original semantic features are the input. Batch size; The intermediate hidden dimension of the first fully connected layer of the node branch; The intermediate hidden dimension of the first fully connected layer of the edge branch; The intermediate hidden layer features are the output of the first layer operation of the node branch; These are the intermediate features after the first layer activation of the edge branch; , The weights and biases of the first fully connected layer; , The weights and biases of the first fully connected layer in the title branch; ReLU is the non-linear activation function; （5）， In the formula, The intermediate hidden layer features are the output of the first layer operation of the node branch; These are the intermediate features after the first layer activation of the edge branch; Intermediate features after Dropout processing of node branches; These are intermediate features after Dropout processing of the edge branches; For random deactivation regularization; Batch size; The intermediate hidden dimension of the first fully connected layer of the node branch; The intermediate hidden dimension of the first fully connected layer of the edge branch; （6）， In the formula, Batch size; The hidden dimension of the fully connected nodes; The hidden dimension of the fully connected edge branch; Intermediate features after Dropout processing of node branches; These are intermediate features after Dropout processing of the edge branches; Features of the final result of node branches; Features of the final result of the edge branch; , The weights and biases for the second fully connected layer; , The weights and biases of the second fully connected layer in the title branch; Dropout is a random deactivation regularization, with The probability is randomly set to zero for some feature values ​​to avoid branch overfitting; The activation function, regularization strategy, and node branches are consistent to ensure the consistency of feature transformation.

4. The method according to claim 3, wherein the method is characterized by: The S3 process is implemented through the following steps: taking text node features as input, a multi-layer mean aggregation GraphSAGE network is constructed to capture the co-occurrence associations between keywords and the spatial structure features of the knowledge graph; the network input is 16-dimensional text node features, which are enhanced by GraphSAGE convolutional layers, and finally output 256-dimensional spatial features. Each intermediate layer introduces ReLU activation function, Dropout regularization and batch normalization. The first The node feature update formula of the layer GraphSAGE is as follows: （7）， wherein is the weight of the edge connecting nodes and ; is the feature vector of node at the layer; is the neighbor aggregated feature vector of node ; （8）， In the formula, It is a node The neighbor aggregated feature vector; The target node In the The feature vector of the layer; It is a vector concatenation operation; It is the first Learnable weight matrix for layer graph convolution; It is the first Learnable bias vectors for layer graph convolution; It is the intermediate feature vector after linear transformation; （9）， In the formula, It is the intermediate feature vector after linear transformation; It is a non-linear activation function, specifically ReLU; It is a node In the The output feature vector after layer graph convolution; This represents random deactivation regularization with a dropout rate of 0.5; Indicates batch normalization; （10）， In the formula, It is the final output feature vector of the entire GraphSAGE module; It is the first Layer node feature vectors; Represents the L2 norm; The CNN-LSTM temporal evolution feature extraction module first concatenates the spatial features of each year's nodes in chronological order to form a temporal feature sequence of the nodes. It then extracts local spatiotemporal correlation features of the sequence using a one-dimensional CNN, employs Same padding to ensure feature dimension consistency, and adjusts the feature sequence dimensions to a format suitable for the LSTM network after batch normalization and ReLU activation. Next, a unidirectional LSTM network captures the long-term temporal dependencies of the sequence, extracting the core features of temporal evolution. Finally, these features are mapped to 256-dimensional temporal features through a fully connected layer and L2 normalized. The fully connected layer transformation maps the LSTM output to the target temporal feature dimension, as shown in the formula: （11）， In the formula, This is the final output temporal feature tensor; For random inactivation regularization; This represents the probability of dropping data in Dropout. The output features of the LSTM at the last time step; For fully connected weights; For bias; For output dimensions; Batch size; Used for CNN-LSTM training, fitting temporal features to spatial features of the last year, formula: （12）， In the formula, This represents the mean squared error loss value; Batch size; This represents the total number of feature dimensions. The predicted temporal features output by the CNN-LSTM network, the first... The node, the first Dimensional features; The real spatial features obtained by encoding in the last year, the first The node, the first Dimensional features; We assign weights of 3.6 and 2.95 to the text features of keyword nodes and the text features of title edges, respectively. Then, we concatenate the weighted text features with spatial and temporal features to obtain the node's fusion features. For each pair of keywords, we extract the fusion features of the two nodes and concatenate them again with the fusion features of the corresponding edges to form the final keyword pair association features. The formula is: （13）， In the formula, is the original graph node feature; is the original graph edge feature; is the scaled graph node feature; is the scaled graph edge feature; , is the fixed scaling coefficient of the feature; （14）， In the formula, The scaled graph node features; These are the static attribute features of the nodes; For node temporal dynamic characteristics; For node-level multimodal fusion features, among which ; For a set of keywords, the node fusion features of each keyword; , , These correspond to the dimensions of nodes, static features, and temporal features, respectively. Batch size; （15）， In the formula, The scaled-down graph edge features; These correspond to the dimensions of nodes, static features, and temporal features, respectively. This represents the final total dimension of the node fusion features; For the final multi-source fusion features, among which ; This represents the total dimension of the final features; Batch size; The hidden dimension of the fully connected edge branch; Using multi-dimensional fused features as input, a weighted random forest classifier is constructed to predict keyword associations; all CPU cores are utilized to improve training and inference efficiency; the decision tree splitting of the random forest is based on the Gini coefficient, formula: （16）， In the formula, is the sample set of the class in the sample set; is the Gini impurity.

5. The method according to claim 4, wherein the method is characterized by: The S4 process is implemented through the following steps: a keyword co-occurrence prediction and technology trend prediction system for the academic field is constructed; the lightweight prediction inference module first performs word matching verification based on the checkpoints of the pre-trained model for the paired keywords input by the user, and directly reuses the pre-calculated feature vectors for the known words already included in the corpus; for the cutting-edge new words and unfamiliar words across fields that are not included, the zero-shot inference mode is started, and the semantic feature extraction is completed in real time through the pre-trained large language model, and finally the real-time calculation of the probability of pairwise keyword association is realized. The technology trend prediction module is based on the annual word frequency statistics of literature keywords and uses the Prophet model to carry out trend prediction. In view of the characteristics of annual literature data, a prior scale of 0.05 for trend change points is set to balance the model's fitting ability and the risk of overfitting. Based on historical year data, the trend projection for the next 5 years is completed. It can quantitatively identify cutting-edge hotspots in their growth phase, declining trends in their decline phase, and basic research areas with steady development; By associating core keywords in the field and consulting with technical personnel in the civil engineering field, we can reasonably infer the development direction of the technology and the related fields that the technology may involve; ultimately, we obtain the Qwen-SGCL intelligent model based on multi-module collaboration.

6. The method according to claim 5, wherein the method is characterized by: The S5 process is implemented through the following steps: Selecting representative baseline models as comparison objects; firstly, traditional machine learning algorithms, including logistic regression, support vector machines, random forests, XGBoost, and AdaBoost; secondly, current mainstream advanced graph learning models, including graph attention networks and graph convolutional networks; employing 10-fold hierarchical cross-validation to ensure robustness of results, repeating experiments and averaging the results; evaluation metrics include: Area under the ROC curve (AUC): （17）， （18）， In the formula, For the first False positive rate per sample; For the first The true positive rate per sample; TP is the number of samples correctly predicted as positive by the model; FP is the number of samples incorrectly predicted as positive by the model; TN is the number of samples correctly predicted as negative by the model; FN is the number of samples incorrectly predicted as negative by the model. Precision: （19）， In the formula, TP represents the number of samples correctly predicted as positive by the model; FP represents the number of samples incorrectly predicted as positive by the model. Recall: （20）， In the formula, TP represents the number of samples correctly predicted as positive by the model; FN represents the number of samples incorrectly predicted as negative by the model. The harmonic mean of precision and recall, F1 score: （21）， Area under the PR curve (AUPR): （22）， In the formula, , For the first , No. Recall rate corresponding to each threshold; , For the first , No. The accuracy corresponding to each threshold.

Images