Scientific and technological supply and demand matching method and equipment based on double-tower model and medium

Abstract

The invention discloses a technology supply and demand matching method and device based on a double-tower model and a medium, and relates to the technical field of information retrieval. The method comprises the following steps: constructing a double-tower model comprising a demand side coding tower and a supply side coding tower; the demand side coding tower adopts a hybrid expert model structure MoE-ALBERT, and the MoE-ALBERT comprises a plurality of expert sub-networks and one dynamic gating mechanism; the supply side encoding tower adopts a hierarchical structure of a text encoder, a table encoder and a cross-modal cross attention fusion layer; carrying out entity identification on the science and technology demand text and the science and technology supply data, linking the science and technology demand text and the science and technology supply data to a science and technology knowledge graph, inputting the preprocessed science and technology demand text into a demand side coding tower for coding to obtain a demand side vector, inputting the preprocessed science and technology supply data into a supply side coding tower for coding to obtain a supply side vector, scientific and technological supply and demand matching is performed according to the similarity score between the demand side vector and the supply side vector, and the problem that heterogeneous data is difficult to process in a scientific and technological resource supply and demand matching scene is solved.

Description

Technical Field

[0001] This invention relates to the field of information retrieval technology, specifically to a technology supply and demand matching method, device, and medium based on a dual-tower model. Background Technology

[0002] In the context of matching supply and demand for science and technology resources, demanders typically present their technical needs in the form of textual descriptions (such as the need for high-efficiency solar cell material preparation technology), while the information provided by suppliers (such as introductions to technical achievements) often includes a large amount of unstructured text (such as technical descriptions and application scenarios) and structured data (such as performance parameter tables and patent information tables).

[0003] In existing technologies, the text of technology demand is usually directly input into a large language model for matching with complete technology supply data. However, the text of technology demand usually exists in the form of natural language text (such as the need for a high-efficiency new energy storage material), while the technology supply data contains a large amount of structured data (such as technical parameter tables, performance index comparison tables, etc.) and unstructured text. Traditional single-modal matching models have difficulty effectively processing such heterogeneous data, resulting in low matching accuracy. Summary of the Invention

[0004] The technical problem to be solved by this invention is how to solve the difficulty of heterogeneous data processing in the scenario of matching supply and demand of scientific and technological resources. The purpose is to provide a method, device and medium for matching supply and demand of scientific and technological resources based on a dual-tower model, thereby solving the above-mentioned problem.

[0005] This invention is achieved through the following technical solution: In a first aspect, the present invention provides a technology supply and demand matching method based on a dual-tower model, comprising: A dual-tower model is constructed, comprising a demand-side encoding tower and a supply-side encoding tower. The demand-side encoding tower adopts a hybrid expert model structure, MoE-ALBERT, which includes multiple expert sub-networks and a dynamic gating mechanism. The supply-side encoding tower adopts a hierarchical structure consisting of a text encoder, a table encoder, and a cross-modal cross-attention fusion layer. Entity recognition is performed on the text of technology demand and the data of technology supply, and the data is linked to the technology knowledge graph to obtain the preprocessed text of technology demand and the preprocessed data of technology supply. The preprocessed technology demand text is input into the demand-side coding tower for encoding to obtain a demand-side vector, and the preprocessed technology supply data is input into the supply-side coding tower for encoding to obtain a supply-side vector. Calculate the similarity score between the demand-side vector and the supply-side vector; Technology supply and demand are matched based on the similarity score.

[0006] Optionally, constructing the dual-tower model includes: Construct demand-side coding towers and supply-side coding towers; Based on a dataset of technological demands, an adversarial training strategy is used to fine-tune the demand-side encoding tower, and a hybrid parameter efficient fine-tuning technique is used to fine-tune the supply-side encoding tower, resulting in a dual-tower model. The hybrid parameter efficient fine-tuning technique involves inserting a LoRA low-rank matrix with a rank of 16 into the Q / K / V projection layer of the Transformer, adding an Adapter module with a bottleneck dimension of 64 into the FeedForward layer, and fine-tuning only the adapter parameters.

[0007] Optionally, the demand-side vector includes a demand semantic vector, a demand domain classification vector, and a demand entity feature vector; the supply-side vector includes a supply semantic vector, a supply domain classification vector, and a supply entity feature vector; calculating the similarity score between the demand-side vector and the supply-side vector includes: Calculate word-level similarity based on the word frequency of keywords in the technology demand text and the technology supply data; Calculate sentence-level similarity based on the demand semantic vector and the supply semantic vector; Calculate domain-level similarity based on the demand domain classification vector and the supply domain classification vector; Calculate entity-level similarity based on the demand entity feature vector and the supply entity feature vector; The word-level similarity, sentence-level similarity, domain-level similarity, and entity-level similarity are weighted and fused to obtain the similarity score between the demand-side vector and the supply-side vector.

[0008] Optionally, the formula for calculating the word-level similarity is as follows: ; Among them, S word The word-level similarity; q i This represents the i-th keyword in the text describing the technology requirement; s j This represents the j-th keyword in the technology supply data; TF-IDF( q i ) represents the frequency of the i-th keyword in the text of the technology requirement; TF-IDF( s j ) represents the frequency of the j-th keyword in the technology supply data; W S ( q i ,s j ) represents the semantic association weight matrix constructed based on the science and technology knowledge graph. q i and s j The weights between them.

[0009] Optionally, the formula for calculating the sentence-level similarity is as follows: ; Among them, S sentence V represents the sentence-level similarity. Q V is the semantic vector of the requirement; S The provided semantic vector; ϵ0 represents a hyperparameter used to control the influence of entropy and confidence on the final similarity, ϵ( Q, S ) represents the information entropy of the matching probability distribution, Entropy(P(Q,S)) is a dynamic adjustment factor that integrates the initial value, uncertainty and model confidence, used to fine-tune the final sentence-level similarity; Confidence(Q,S) is the confidence score of the dual-tower model for the current match.

[0010] Optionally, the formula for calculating the domain-level similarity is as follows: ; Among them, S domain G(Q) represents the domain-level similarity; G(S) represents the demand domain classification vector; G(S) represents the supply domain classification vector; cos() represents the cosine similarity.

[0011] Optionally, the formula for calculating the entity-level similarity is as follows: ; Among them, S entity E represents the entity-level similarity. Q E represents the feature vector of the demand entity; S E represents the feature vector of the supply entity; Q ∩E S |E represents an entity that includes both supply and demand. Q ∪E S | represents the total number of all non-repeating entities in demand and supply; e is E Q With E S Any entity in the intersection of; Sim(e Q ,e S ) is the entity pair e Q and e S Semantic similarity between them; e Q Describe the characteristics of entity e on the demand side, e SThis represents the characteristics of entity e in the supply side.

[0012] Optionally, the weights used in the weighted fusion are obtained in the following way: ; in, λ i The weight of the i-th similarity among the word-level similarity, sentence-level similarity, domain-level similarity, and entity-level similarity; I i (Q,S) represents the importance coefficient of the i-th similarity, obtained through dynamic learning of gradients using a deep deterministic strategy; exp represents the exponential function; I k (Q,S) is the importance coefficient of the k-th similarity, k=1,2,3,4; α k I represents the importance coefficient of the k-th similarity. k Scaling factor of (Q,S); α i I represents the importance coefficient of the i-th similarity. i Scaling factor of (Q,S).

[0013] In a second aspect, the present invention provides a computer device, the computer device including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement a technology supply and demand matching method based on a dual-tower model as described in any one of the first aspects.

[0014] Thirdly, the present invention provides a computer storage medium storing a computer program, wherein the computer program is executed by a processor to implement a technology supply and demand matching method based on a dual-tower model as described in any one of the first aspects.

[0015] Compared with the prior art, the present invention has the following advantages and beneficial effects: This application provides a technology supply and demand matching method based on a dual-tower model. This method constructs a dual-tower model adapted to heterogeneous data, comprising a demand-side encoding tower and a supply-side encoding tower. The demand-side encoding tower adopts a hybrid expert model structure, MoE-ALBERT, which includes multiple expert sub-networks and a dynamic gating mechanism. The supply-side encoding tower employs a hierarchical structure of a text encoder, a table encoder, and a cross-modal cross-attention fusion layer. The demand-side encoding tower encodes the technology demand text to obtain a demand-side vector, and the supply-side encoding tower encodes the technology supply data (encoding the text portion of the technology supply data through a text encoder and the table portion through a table encoder) to obtain a supply-side vector. This separate approach processes technology demand text and technology supply data with different data structures, solving the problem of handling heterogeneous data in technology resource supply and demand matching scenarios, thereby improving the encoding accuracy of technology demand text and technology supply data. Finally, technology supply and demand matching is performed based on the similarity score between the demand-side vector and the supply-side vector, further improving the accuracy of technology supply and demand matching. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 A flowchart illustrating the technology supply and demand matching method based on a dual-tower model provided in this application embodiment; Figure 2 This is a schematic diagram of a twin-tower model provided in an embodiment of this application; Figure 3 A multi-scale similarity calculation framework diagram provided for embodiments of this application; Figure 4 A schematic diagram of the training process of a dual-tower model provided in an embodiment of this application; Figure 5 This is a structural diagram of a technology supply and demand matching system based on a dual-tower model, provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0018] In existing technologies, the text of technology demand is usually directly input into a large language model for matching with complete technology supply information, but this has the following problems: 1. Data heterogeneity problem: Technology demand texts are usually in the form of natural language text, while technology supply information contains a large amount of structured data and unstructured text. Traditional single-modality matching models are difficult to effectively handle such heterogeneous data.

[0019] 2. Token length limit issue: As the mainstream technology in current natural language processing, large language models have a limit on the number of input tokens (the smallest units into which text is broken down). When processing technology supply information containing complex tables, fully parsing the table structure will quickly exhaust the token quota, causing large models to be unable to process long texts or complex structured data.

[0020] 3. Insufficient matching accuracy: Existing keyword-based or traditional machine learning-based matching methods struggle to understand the deep semantic relationships between technology demand and supply, leading to a semantic gap. For example, energy density in demand and specific capacity in supply tables are actually related concepts, but traditional methods cannot recognize such semantic correspondences within specialized fields.

[0021] 4. Matching efficiency issues: The database of science and technology resources is huge, containing tens of thousands to millions of supply information entries. The traditional method of comparing demand and supply one by one is computationally complex and cannot meet the needs of real-time matching.

[0022] 5. Interpretability and robustness issues: Most existing deep learning models are black box systems, lacking an interpretation mechanism for matching results, and have insufficient generalization ability when dealing with noisy data or domain transfer scenarios.

[0023] Therefore, this application provides a technology supply and demand matching method based on a dual-tower model. Please refer to... Figure 1 This is a flowchart illustrating the technology supply and demand matching method based on a dual-tower model provided in this application embodiment. The following is a combination of... Figure 1 Let me introduce it.

[0024] S101. Construct a dual-tower model.

[0025] The dual-tower model comprises a demand-side encoding tower and a supply-side encoding tower. The demand-side encoding tower employs a hybrid expert model structure, MoE-ALBERT, which includes multiple expert sub-networks (e.g., 16 expert sub-networks) and a dynamic gating mechanism. The supply-side encoding tower uses a hierarchical structure of a text encoder, a table encoder, and a cross-modal cross-attention fusion layer. The text encoder also uses a hybrid expert model structure (MoE-ALBERT), while the table encoder uses the FLAN-T5-XL model. MoE-ALBERT supports an 8192-tower token context window, which can solve the token input overload problem commonly found in traditional large models.

[0026] In one possible embodiment, the specific steps of S101 include: Demand-side and supply-side encoding towers are constructed. Based on a demand dataset in the technology field, an adversarial training strategy is used to fine-tune the demand-side encoding tower, and a hybrid parameter efficient fine-tuning technique is used to fine-tune the supply-side encoding tower, resulting in a dual-tower model. The hybrid parameter efficient fine-tuning technique involves inserting a LoRA low-rank matrix with a rank of 16 into the Q / K / V projection layer of the Transformer, adding an Adapter module with a bottleneck dimension of 64 into the FeedForward layer, and fine-tuning only the adapter parameters.

[0027] Specifically, firstly, a demand-side encoding tower is constructed using the hybrid expert model structure MoE-ALBERT (containing 16 expert sub-networks and 1 gating mechanism). Then, a supply-side encoding tower is constructed using a hierarchical structure (MoE-ALBERT text encoder, FLAN-T5-XL table encoder, and cross-modal cross-attention fusion layer). Next, the demand-side encoding tower is fine-tuned using a technology domain demand dataset to optimize demand semantic extraction capabilities, and an adversarial training strategy (FGSM attack) is employed to improve model robustness. Finally, a hybrid parameter efficient fine-tuning technique is used to fine-tune the supply-side encoding. This hybrid parameter efficient fine-tuning technique includes the following aspects: Inserting a low-rank LoRA matrix (r=16) into the Q / K / V projection layer of the Transformer, the optimal rank was determined to be 16 through comparative experiments (r=8 / 16 / 32), which improved the F1-score by 2.3% on the validation set.

[0028] Add an Adapter module (bottleneck dimension = 64) to the FeedForward layer and determine the optimal bottleneck dimension through grid search (32 / 64 / 128).

[0029] 3. Fine-tune the adapter parameters only, set the learning rate to 1e-4, the batch size to 4, and use mixed precision training.

[0030] 4. Table column selection model: Filter relevant column data in the supply table based on demand semantic vector.

[0031] 5. Table row selection model: Further filter relevant row data based on the filtered column data.

[0032] In one possible implementation, the dual-tower model can be compressed using a structured compression strategy that combines reinforcement learning with self-supervised pre-training.

[0033] ; Among them, C R Indicates compression ratio; L original L is the length of the token sequence after converting the original table text. compressed This represents the length of the compressed token sequence.

[0034] The reward function is as follows: ; Among them, R total For the cumulative reward function, R C To correctly select the reward value for the relevant data, P I Penalty value for incorrectly selecting irrelevant data. P M The penalty value for missing relevant data. R E Rewards are retained for entity information, where ϵ is the entity weight coefficient, with an optimal value of 0.8; α, β, and γ are the weight coefficients of each reward, where α > β > γ ensures the model prioritizes correct matching; I imp The unit penalty value for selecting the wrong penalty item; M imp C is the unit penalty value for omissions; R Compression ratio; is the weighting coefficient for compression ratio; log represents the logarithmic function.

[0035] In one possible implementation, the finely tuned dual-tower model parameters can be compressed from FP32 to INT8 using a symmetric quantization method, achieving a 75% reduction in GPU memory usage and a 2.3x increase in inference speed in an experimental environment.

[0036] S102. Perform entity recognition on the technology demand text and technology supply data, and link them to the technology knowledge graph to obtain the preprocessed technology demand text and preprocessed technology supply data.

[0037] Among them, technology demand text refers to textual content describing problems, challenges, needs, or goals in the fields of science, research, or technology, specifically in the form of natural language text. Technology supply data refers to textual or data content describing technologies, resources, services, or solutions provided in the fields of science, research, or technology, including both text and tabular portions.

[0038] Specifically, the process begins by acquiring textual data on technology demand and supply. Then, the technology supply data is parsed to extract table structure, including metadata and content data. The metadata comprises table headers, row and column relationships, and data types. Next, a bidirectional encoder-representation-bidirectional long short-term memory-conditional random field (BERT-BiLSTM-CRF) model, pre-trained on a corpus of science and technology data, is used to perform entity recognition on the technology demand text and supply data, i.e., labeling entities in the science and technology field (such as material names, technical indicators, application scenarios, etc.). Finally, the labeled entities are linked to a self-built science and technology knowledge graph.

[0039] S103. Input the preprocessed technology demand text into the demand-side coding tower for encoding to obtain the demand-side vector, and input the preprocessed technology supply data into the supply-side coding tower for encoding to obtain the supply-side vector.

[0040] Please refer to Figure 2 This is a schematic diagram of a dual-tower model provided in an embodiment of this application. The dual-tower model includes a demand-side coding tower and a supply-side coding tower, which will be discussed below. Figure 2 This section describes the specific processing flow of the twin-tower model.

[0041] Specifically, the technology requirement text is input into the demand-side coding tower. Through a gating mechanism and 16 expert sub-networks, demand-side vectors are output, including a demand semantic vector, a demand domain classification vector, and a demand entity feature vector. Among them, the demand semantic vector indicates the overall semantic information of the technology requirement text, the demand domain classification vector indicates the technical field to which the technology requirement text belongs, and the demand entity feature vector indicates the features of key technical entities in the technology requirement text.

[0042] Technology supply data is input into the supply-side encoding tower. The MoE-ALBERT text encoder encodes the text portion to obtain text vectors, and the FLAN-T5-XL table encoder encodes the table portion to obtain table vectors. Finally, the text vectors and table vectors are input together into a cross-modal cross-attention fusion layer. The text vectors and table vectors are fused through a cross-attention mechanism to output supply-side vectors, including supply semantic vectors, supply domain classification vectors, and supply entity feature vectors. Among them, the supply semantic vector indicates the overall semantic information of the technology supply data, the supply domain classification vector indicates the technical field to which the technology supply data belongs, and the supply entity feature vector indicates the features of key technical entities in the technology supply data.

[0043] S104. Calculate the similarity score between the demand-side vector and the supply-side vector.

[0044] Please refer to Figure 3 This is a framework diagram for multi-scale similarity calculation provided in the embodiments of this application. The following is in conjunction with... Figure 3 The specific steps for implementing S104 are as follows: S1.1 Calculate word-level similarity based on the word frequency of keywords in the science and technology demand text and science and technology supply data.

[0045] Specifically, the formula for calculating word-level similarity is as follows: ; Among them, S word Word-level similarity; q i This represents the i-th keyword in the technology requirement text (i.e., the i-th requirement keyword); s j This represents the j-th keyword (i.e., the i-th supply keyword) in the technology supply data; TF-IDF( q i ) represents the frequency of the i-th keyword in the text of technology requirements; TF-IDF( s j ) represents the frequency of the j-th keyword in the technology supply data; W S ( q i , s j ) represents the semantic association weight matrix constructed based on the science and technology knowledge graph. q i and s j The weights between them.

[0046] S1.2 Calculate sentence-level similarity based on the demand semantic vector and the supply semantic vector.

[0047] Specifically, the formula for calculating sentence-level similarity is as follows: ; Among them, S sentence Sentence-level similarity; V Q V is the semantic vector of demand; S To provide semantic vectors; ϵ0 represents a hyperparameter used to control the influence of entropy and confidence on the final similarity, ϵ( Q, S ) represents the information entropy of the matching probability distribution, and Entropy(P(Q,S)) is a dynamic adjustment factor that integrates the initial value, uncertainty and model confidence, used to fine-tune the final similarity calculation result; Confidence(Q,S) is the confidence score of the dual-tower model for the current match.

[0048] S1.3 Calculate domain-level similarity based on the demand domain classification vector and the supply domain classification vector.

[0049] The formula for calculating domain-level similarity is as follows: ; Among them, S domain G(Q) represents the domain-level similarity; G(Q) is the demand domain classification vector, generated by a domain classifier; G(S) is the supply domain classification vector, generated by a domain classifier; cos() represents the cosine similarity.

[0050] S1.4 Calculate entity-level similarity based on the feature vectors of demand entities and supply entities.

[0051] The formula for calculating entity-level similarity is as follows: ; Among them, S entity E represents the entity-level similarity. Q E represents the feature vector of the demand entity; S E represents the feature vector of the supply entity; Q ∩E S |E represents an entity that includes both supply and demand. Q ∪E S | represents the total number of all non-repeating entities in demand and supply; e is E Q With E S Any entity in the intersection of; Sim(e Q ,e S ) is the entity pair e Q and e S Semantic similarity between them; e Q Describe the characteristics of entity e on the demand side, e S Sim(e) represents the characteristics of entity e in the supply side.Q ,e S The calculation method for ) is as follows: Sim(e Q ,e S = 0.7Simpath + 0.3Simembed; Simpath represents the path similarity of the science and technology knowledge graph, and Simembed represents the cosine similarity of entity embeddings.

[0052] S1.5. Weighted summation of word-level similarity, sentence-level similarity, domain-level similarity, and entity-level similarity to obtain the similarity score between the demand-side vector and the supply-side vector.

[0053] Specifically, the formula for calculating the similarity score is as follows: ; Where S(Q, S) represents the similarity score between the demand-side vector and the supply-side vector; S word For word-level similarity; S sentence S represents sentence-level similarity; domain Domain-level similarity; S entity λ1 represents entity-level similarity; λ2, λ3, and λ4 are the dynamic learning weights for each similarity level, obtained as follows: ; in, λ i This represents the weight of the i-th similarity among the four similarity metrics (word-level similarity, sentence-level similarity, domain-level similarity, and entity-level similarity). i =1,2,3,4; exp represents the exponential function; I i (Q,S) represents the importance coefficient of the i-th similarity, obtained through dynamic learning via Deep Deterministic Policy Gradient (DDPG); the DDPG state space contains 4 similarities, historical Top-100 matching accuracy, and current domain classification confidence, totaling 10-dimensional feature vectors. k (Q,S) is the importance coefficient of the k-th similarity, k=1,2,3,4; α k I represents the importance coefficient of the k-th similarity. k Scaling factor of (Q,S); α i I represents the importance coefficient of the i-th similarity. i Scaling factor of (Q,S).

[0054] S105. Match technology supply and demand based on similarity scores.

[0055] Specifically, if the similarity score is greater than or equal to a preset threshold (e.g., 0.85), it is determined that the technology demand text and the technology supply data are highly matched, and an explanation text based on the matching criteria is generated. If the similarity score is less than the preset threshold (e.g., 0.85), it is determined that the technology demand text and the technology supply data are not matched.

[0056] To more clearly illustrate the technology supply and demand matching method based on the dual-tower model provided in this application, the following example is given.

[0057] (I) Data Preprocessing Examples Taking typical high-temperature material requirements as an example, the technical requirement text might read: "A ceramic thermal insulation material with high temperature resistance >1000℃, thermal conductivity <20W / (m・K), and flexural strength >300MPa is required." The preprocessing steps are as follows: S1.1 Word segmentation: A specialized word segmentation tool for the scientific and technological field is used to identify technical terms and numerical ranges.

[0058] S1.2 Text standardization: unify unit representation and remove redundant information.

[0059] S1.3 Entity Recognition and Linking Process: Entity annotation is performed using a BERT-BiLSTM-CRF model pre-trained in the technology field, identifying core entities such as ceramic thermal insulation materials (material entities) and high-temperature resistance (performance entities); entity linking technology is used to link to the technology knowledge graph to automatically obtain the attribute features and relationship network of the entities.

[0060] S1.4 Keyword Extraction: Use Term Frequency-Inverse Document Frequency (TF-IDF) and TextRank algorithms to extract core requirement elements, i.e., keywords.

[0061] For example, if the technology supply data is a technical document and performance parameter table for a certain ceramic material, the preprocessing steps are as follows: S2.1 Text processing: Similar to the preprocessing of the requirement data, the focus is on extracting technical features, application scenarios and entity information.

[0062] S2.2 Table Processing: First, the table header level, data type, and unit information are identified. The parsing algorithm for merged cells and multi-level headers is optimized. The logical row and column relationships are reconstructed by recursively traversing the table DOM structure. Then, missing values ​​and outliers are handled. Finally, the table data is associated with text entities to establish a cross-modal entity index.

[0063] (II) Model Construction Implementation Examples (1) Basic model building: Demand-side coding tower: adopts MoE-ALBERT, 16 expert sub-networks, each expert contains 18 Transformer layers, 4096 hidden dimensions, and a total of 312M parameters.

[0064] The text encoder for the supply-side coding tower: adopts MoE-ALBERT (sharing weights with the demand-side coding tower).

[0065] The table encoder of the supply-side coding tower: adopts FLAN-T5-XL, 24-layer Transformer, 1024 hidden dimensions, and 3B parameters.

[0066] Cross-modal attention layer: 8-head self-attention, dropout=0.1.

[0067] Output vector dimension: uniformly set to 4096 dimensions.

[0068] (2) MoE-ALBERT training strategy: Please refer to Figure 4 This is a schematic diagram of the training process of a dual-tower model provided in an embodiment of this application.

[0069] The gate control mechanism is as follows: ; in, E i ( x ) is the first i The output of the expert subnetwork G i (x) represents the gate weights, and softmax represents the normalization function; W g Let b represent the weight matrix of the gated network. g Let y represent the bias vector of the gated network, x represent the input data, and y represent the weighted sum of the outputs of all expert subnetworks.

[0070] The expert selection strategy is optimized using a load balancing loss function, allowing each expert sub-network to focus on addressing the specific needs of a particular technical field. The loss function is as follows: ; Among them, L gate To mitigate the losses associated with expert load balancing, load balancing is achieved by minimizing the variance of the activation frequencies of each expert. and As balance coefficients, the optimal values ​​were determined to be 0.3 and 0.7 through grid search. MLM The loss is the standard masked language model loss, ensuring the model can guess the masked word; L SOP The loss is used to predict sentence order, allowing the model to learn the sequential relationships between sentences; LMoE The total loss of the hybrid expert model is obtained by weighted summation of the first two losses and the load balancing loss.

[0071] Add FGSM perturbation to the input embedding layer: ; Where, δ FGSM This indicates the generated adversarial perturbation; sign represents the sign function for taking the gradient direction. L(x,y) The loss function representing the model; ϵ represents the gradient of the input sample x, used to indicate the direction of the fastest change in loss; ϵ represents the magnitude coefficient controlling the perturbation. Adversarial examples comprise 20% of the training samples.

[0072] (3) Implementation of hierarchical table coding: LoRA configuration: rank r=16, scaling factor α=32.

[0073] Adaptation layer location: Q / K / V attention projection layer.

[0074] Adapter configuration: bottleneck dimension = 64, activation function = GELU.

[0075] Location: After the feedforward layer of each Transformer block.

[0076] Fine-tuning the objective function: ; in, For the total loss, To match the loss, To minimize losses, For regularization loss, For entity loss, α, β, γ and δ represent the weights of each loss.

[0077] (III) Matching Reasoning Examples (1) Milvus vector retrieval implementation: Index type: IVF1024, Flat (inverted index + Flat quantization).

[0078] Training data: 1 million randomly sampled supply vectors.

[0079] Query parameter: nprobe=32 (retrieves 32 cluster centers).

[0080] Batch processing: Supports parallel processing of 100 queries per batch.

[0081] (2) Explanation of the generation implementation: Attention visualization: Extract the attention weights of the last layer and generate attention heatmaps of demand text and supply text.

[0082] Entity Matching Explanation: Lists the top 100 matching entity pairs and their similarity scores.

[0083] The high matching degree is mainly due to the matching of the ceramic insulation material entities (similarity 0.92) and the matching of thermal conductivity performance parameters (similarity 0.88).

[0084] (3) Performance indicators: Compression rate: 86% (original token count: 4800 → compressed token count: 672), entity retention rate: 98.7%.

[0085] Matching accuracy: 93.8% precision, 92.6% recall, and 93.2% F1 score.

[0086] Response speed: Average time per match is 0.52 seconds (INT8 quantization + Milvus retrieval).

[0087] Scalability: Supports real-time retrieval of supply data at the level of 10 million.

[0088] In summary, this application provides a technology supply and demand matching method based on a dual-tower model, achieving technological breakthroughs through three core innovations: first, constructing a dual-tower model architecture adapted to heterogeneous data; second, developing a parameter-efficient fine-tuning compression technology; third, designing a dynamic multi-scale similarity calculation framework; and fourth, constructing an interpretable retrieval framework. Ultimately, it achieves millisecond-level accurate matching of technology supply and demand information, with the following beneficial effects: 1. Enhanced multi-scale semantic understanding: Multi-scale similarity calculation improves the matching F1-score to 93.2%.

[0089] 2. Dynamic adaptability and robustness: Through MoE structure and adversarial training, the accuracy retention rate is improved by 15% in noisy data scenarios.

[0090] 3. High-efficiency long text processing: The MoE structure supports 8192 tokens in the context window, and combined with INT8 quantization, the inference speed is improved by 2.3 times.

[0091] 4. Intelligent structured compression: A compression strategy that integrates entity preservation mechanisms achieves an entity information retention rate of 98.7% at a compression rate of 86%.

[0092] 5. Deep integration of domain knowledge: Entity linking and knowledge graph embedding technology improve the accuracy of professional terminology matching by 21%.

[0093] 6. Enhanced interpretability: Attention visualization and entity matching explanation significantly enhance the transparency of model decision-making.

[0094] 7. Large-scale deployment capability: Milvus vector search supports tens of thousands of queries per second, making it suitable for ultra-large-scale technology resource database applications.

[0095] Please refer to Figure 5 The following is a structural diagram of a technology supply and demand matching system based on a dual-tower model provided in an embodiment of this application. The functions of each module in the system are described below.

[0096] Data Acquisition Layer: Used to acquire technology demand text and technology supply data (text and tables); Data preprocessing layer: used for data cleaning, format conversion, structured parsing, and entity recognition and linking; Dual-tower model layer: includes demand-side coding tower and supply-side coding tower, which respectively process technology demand text and technology supply data, and output demand semantic vector, domain classification vector and entity feature vector; Model training module: used for model parameter optimization and fine-tuning, employing hybrid parameter efficient fine-tuning techniques and adversarial training; Multi-scale matching module: Enables dynamic fusion of word-level similarity, sentence-level similarity, domain-level similarity, and entity-level similarity; Vector retrieval module: Enables fast retrieval of large-scale supply vectors based on Milvus; Explanation Generation Module: Generates visual explanations of the matching results; Results Display Layer: Used to visually display matching results, similarity scores, and explanatory text.

[0097] Based on the same inventive concept, this application also provides a computer device, which includes a processor, a memory, and a computer program stored in the memory. The computer program is executed by the processor to implement the aforementioned technology supply and demand matching method based on the dual-tower model.

[0098] Based on the same inventive concept, this application also provides a computer storage medium storing a computer program, which is executed by a processor to implement the aforementioned technology supply and demand matching method based on the dual-tower model.

[0099] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a device including one or any combination of the above-mentioned memories. The computer may be a variety of computing devices, including smart terminals and servers.

[0100] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0101] As an example, executable instructions may, but do not necessarily, correspond to files in the file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborative files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0102] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0103] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0104] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0105] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A technology supply and demand matching method based on a dual-tower model, characterized in that, include: A dual-tower model is constructed, comprising a demand-side encoding tower and a supply-side encoding tower. The demand-side encoding tower adopts a hybrid expert model structure, MoE-ALBERT, which includes multiple expert sub-networks and a dynamic gating mechanism. The supply-side encoding tower adopts a hierarchical structure consisting of a text encoder, a table encoder, and a cross-modal cross-attention fusion layer. Entity recognition is performed on the text of technology demand and the data of technology supply, and the data is linked to the technology knowledge graph to obtain the preprocessed text of technology demand and the preprocessed data of technology supply. The preprocessed technology demand text is input into the demand-side coding tower for encoding to obtain a demand-side vector, and the preprocessed technology supply data is input into the supply-side coding tower for encoding to obtain a supply-side vector. Calculate the similarity score between the demand-side vector and the supply-side vector; Technology supply and demand are matched based on the similarity score.

2. The technology supply and demand matching method based on the dual-tower model according to claim 1, characterized in that, The construction of the dual-tower model includes: Construct demand-side coding towers and supply-side coding towers; Based on a dataset of technological demands, an adversarial training strategy is used to fine-tune the demand-side encoding tower, and a hybrid parameter efficient fine-tuning technique is used to fine-tune the supply-side encoding tower, resulting in a dual-tower model. The hybrid parameter efficient fine-tuning technique involves inserting a LoRA low-rank matrix with a rank of 16 into the Q / K / V projection layer of the Transformer, adding an Adapter module with a bottleneck dimension of 64 into the FeedForward layer, and fine-tuning only the adapter parameters.

3. The technology supply and demand matching method based on the dual-tower model according to claim 1, characterized in that, The demand-side vector includes a demand semantic vector, a demand domain classification vector, and a demand entity feature vector; the supply-side vector includes a supply semantic vector, a supply domain classification vector, and a supply entity feature vector; calculating the similarity score between the demand-side vector and the supply-side vector includes: Calculate word-level similarity based on the word frequency of keywords in the technology demand text and the technology supply data; Calculate sentence-level similarity based on the demand semantic vector and the supply semantic vector; Calculate domain-level similarity based on the demand domain classification vector and the supply domain classification vector; Calculate entity-level similarity based on the demand entity feature vector and the supply entity feature vector; The word-level similarity, sentence-level similarity, domain-level similarity, and entity-level similarity are weighted and fused to obtain the similarity score between the demand-side vector and the supply-side vector.

4. The technology supply and demand matching method based on the dual-tower model according to claim 3, characterized in that, The formula for calculating word-level similarity is as follows: ； Among them, S word The word-level similarity; q i This represents the i-th keyword in the text describing the technology requirement; s j This represents the j-th keyword in the technology supply data; TF-IDF( q i ) represents the frequency of the i-th keyword in the text of the technology requirement; TF-IDF( s j ) represents the frequency of the j-th keyword in the technology supply data; W S ( q i , s j ) represents the semantic association weight matrix constructed based on the science and technology knowledge graph. q i and s j The weights between them.

5. The technology supply and demand matching method based on the dual-tower model according to claim 3, characterized in that, The formula for calculating sentence-level similarity is as follows: ； Among them, S sentence V represents the sentence-level similarity. Q V is the semantic vector of the requirement; S The provided semantic vector; ϵ0 represents a hyperparameter used to control the influence of entropy and confidence on the final similarity; ϵ( Q, S ) represents the information entropy of the matching probability distribution, Entropy(P(Q,S)) is a dynamic adjustment factor that integrates the initial value, uncertainty and model confidence, used to fine-tune the final sentence-level similarity; Confidence(Q,S) is the confidence score of the dual-tower model for the current match.

6. The technology supply and demand matching method based on the dual-tower model according to claim 3, characterized in that, The formula for calculating the domain-level similarity is as follows: ； Among them, S domain G(Q) represents the domain-level similarity; G(S) represents the demand domain classification vector; G(S) represents the supply domain classification vector; cos() represents the cosine similarity.

7. The technology supply and demand matching method based on the dual-tower model according to claim 3, characterized in that, The formula for calculating entity-level similarity is as follows: ； Among them, S entity The entity-level similarity; Sim(e Q ,e S ) is the entity pair e Q and e S Semantic similarity between them; E Q E represents the feature vector of the demand entity; S E represents the feature vector of the supply entity; Q ∩E S |E represents an entity that includes both supply and demand. Q ∪E S | represents the total number of all non-repeating entities in demand and supply; e is E Q With E S any entity in the intersection of; e Q Describe the characteristics of entity e on the demand side, e S This represents the characteristics of entity e in the supply side.

8. The technology supply and demand matching method based on the dual-tower model according to claim 3, characterized in that, The weights used in the weighted fusion are obtained in the following way: ； in, λ i The weight of the i-th similarity among the word-level similarity, sentence-level similarity, domain-level similarity, and entity-level similarity; I i (Q,S) represents the importance coefficient of the i-th similarity, obtained through dynamic learning of gradients using a deep deterministic strategy; exp represents the exponential function; I k (Q,S) is the importance coefficient of the k-th similarity, k=1,2,3,4; α k I represents the importance coefficient of the k-th similarity. k Scaling factor of (Q,S); α i I represents the importance coefficient of the i-th similarity. i Scaling factor of (Q,S).

9. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement a technology supply and demand matching method based on a dual-tower model as described in any one of claims 1-8.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements a technology supply and demand matching method based on a dual-tower model as described in any one of claims 1-8.

Images