A method for adapting enterprise scientific and technological achievements based on accurate retrieval and query of big data

Abstract

The present invention discloses a method for adapting enterprise scientific and technological achievements based on precise retrieval and query of big data, which relates to the field of big data processing and knowledge graph technology. The method includes constructing an enterprise technical field knowledge graph based on a standardized multimodal feature matrix, using the Apache Flink streaming processing framework to receive incremental technical data in real time, dynamically updating the graph node relationship, removing expired nodes through a pruning algorithm, and outputting a dynamic knowledge graph with a timestamp; receiving a user query request, extracting a query vector using the cross-modal embedding model, retrieving a list of Top-N candidate nodes through the dynamic knowledge graph, calculating a comprehensive similarity score based on the graph relationship weights, and outputting a sorted list of technical achievements. By mapping multi-source heterogeneous data to a unified semantic space through a cross-modal embedding model and combining it with the construction and update of a dynamic knowledge graph, the intelligent level and practical application effect of scientific and technological achievement adaptation are comprehensively improved.

Description

Technical Field

[0001] The present invention relates to the field of big data processing and knowledge graph technology, and in particular to a method for adapting enterprise scientific and technological achievements based on precise retrieval and query of big data. Background Art

[0002] With the rapid development of big data technology, the management and retrieval of corporate scientific and technological achievements have become critical to improving corporate innovation capabilities and the efficiency of technology transformation. Traditional technology retrieval methods rely primarily on keyword matching and structured database queries. While this approach performs well when dealing with a single data type, its limitations become increasingly apparent when dealing with multi-source, heterogeneous data.

[0003] When generating adaptation solutions, existing technologies lack the integration and constraint design of multi-objective optimization algorithms, making it difficult to simultaneously meet the requirements of multiple optimization objectives, which limits the practical application of adaptation solutions. There is an urgent need for a technical solution that can effectively integrate multimodal data, dynamically update knowledge graphs, and integrate multi-objective optimization algorithms to improve the efficiency and accuracy of enterprise scientific and technological achievement retrieval and adaptation. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an enterprise scientific and technological achievement adaptation method based on big data precise retrieval and query to solve the problem that the adaptation scheme is difficult to meet multiple optimization target requirements at the same time.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides an enterprise scientific and technological achievement adaptation method based on big data precise retrieval and query, which comprises: collecting enterprise multi-source heterogeneous data and preprocessing it to generate a standardized multimodal enterprise data set;

[0008] The standardized multimodal enterprise dataset is mapped to a unified semantic space through a cross-modal embedding model to generate a standardized multimodal feature matrix.

[0009] Based on a standardized multimodal feature matrix, we build a knowledge graph for the enterprise's technical field. We use the Apache Flink streaming framework to receive incremental technical data in real time, dynamically update graph node relationships, and remove expired nodes through a pruning algorithm to output a dynamic knowledge graph with timestamps.

[0010] Receive user query requests, extract query vectors using the cross-modal embedding model, retrieve the Top-N candidate node list through the dynamic knowledge graph, calculate the comprehensive similarity score based on the graph relationship weights, and output a ranked list of technical achievements;

[0011] Using the sorted list of technical achievements, an adaptation plan is generated by integrating multi-objective constraints through a multi-objective optimization algorithm.

[0012] As a preferred solution of the enterprise scientific and technological achievement adaptation method based on big data precise retrieval and query described in the present invention, the steps of collecting enterprise multi-source heterogeneous data and preprocessing it to generate a standardized multimodal enterprise data set are as follows:

[0013] Collect R&D project data, technical parameter tables, technical drawings and design documents to generate internal enterprise data, and collect industry data, supply chain data, industry reports and market analysis data to generate external enterprise data and obtain multi-source heterogeneous enterprise data;

[0014] Use interpolation to fill missing values, remove duplicates from unique identifier fields, use regular expressions to filter invalid characters, and use statistical methods to detect and handle outliers;

[0015] Unify the units and time formats of structured data, convert unstructured data into a unified format, and encode categorical data to generate standardized multimodal enterprise datasets.

[0016] As a preferred solution of the enterprise scientific and technological achievement adaptation method based on big data precise retrieval and query described in the present invention, wherein: the standardized multimodal enterprise dataset is mapped to a unified semantic space through a cross-modal embedding model to generate a standardized multimodal feature matrix, the specific steps are as follows:

[0017] Based on a historical standardized multimodal enterprise dataset, a multimodal training dataset is constructed. The contrastive loss function is used to minimize the embedding distance of positive sample pairs and maximize the embedding distance of negative sample pairs to train the CLIP model to learn the semantic alignment of multimodal data.

[0018] Based on the standardized multimodal enterprise dataset, the trained CLIP model is mapped to a unified semantic space, and the weight ratios of different modalities are dynamically adjusted to form a standardized multimodal feature matrix.

[0019] As a preferred solution of the enterprise scientific and technological achievements adaptation method based on big data precise search and query described in the present invention, the enterprise technology field knowledge graph is constructed based on the standardized multimodal feature matrix, as follows:

[0020] Based on the enterprise's multi-source heterogeneous data, technical entities, enterprise entities, and external entities are extracted, and based on the explicit associations between the enterprise's multi-source heterogeneous data, three types of relationships are defined: technology, enterprise, and external;

[0021] Merge the same entity in heterogeneous data sources from different enterprises into a unique node through entity matching algorithms, and standardize the same relationship through relationship mapping rules;

[0022] Neo4j is used to store entities and relationships as knowledge graph nodes and edges, and the entity semantic vectors and cross-modal association information in the standardized multimodal feature matrix are used as node attributes and relationship weights to obtain the enterprise technology field knowledge graph.

[0023] As a preferred solution of the enterprise scientific and technological achievement adaptation method based on big data precise retrieval and query described in the present invention, the method includes: real-time access to incremental technical data streams through Apache Flink, parsing and extracting entities, relationships and timestamps, and processing out-of-order events based on event time distribution and watermarks. A dynamic update strategy is used to aggregate operations within the time window, and batch writes are made to Neo4j through asynchronous I / O. At the same time, expired nodes in the enterprise technology field knowledge graph are pruned based on node activity tags and business rules, Cypher deletion operations are performed regularly, and finally a dynamic knowledge graph with a timestamp is output.

[0024] As a preferred solution of the enterprise scientific and technological achievement adaptation method based on big data precise retrieval query described in the present invention, wherein: receiving a user query request, extracting a query vector using the cross-modal embedding model, and retrieving a Top-N candidate node list through a dynamic knowledge graph, the specific steps are as follows:

[0025] Use the Flask framework to build an API interface, define the JSON format of query requests, receive user query requests, parse user query requests through natural language processing tools, extract query keywords and verify the legitimacy of parameters;

[0026] The trained CLIP model maps query keywords to a unified semantic space and uses a text encoder to output a query vector.

[0027] Based on the dynamic knowledge graph, FAISS is used to index all node vectors, and the search method is used to retrieve the Top-N candidate nodes. The semantic similarity scores between the query vector and some node vectors are calculated through the approximate nearest neighbor algorithm, and the Top-N candidate node list is output.

[0028] As an optimal solution of the enterprise scientific and technological achievements adaptation method based on big data precise retrieval query described in the present invention, the following is described: based on the Top-N candidate node list, the relationship weights of the Top-N candidate nodes are queried from the knowledge graph, the Top-N candidate nodes and the relationship weights are output, the semantic similarity scores are adjusted according to the relationship weights to obtain a comprehensive similarity score, and the scores are sorted in descending order, the Top-K results are intercepted, and the sorted list of technical achievements is output.

[0029] As a preferred solution of the enterprise scientific and technological achievement adaptation method based on big data precise retrieval and query of the present invention, wherein: the adaptation solution is generated by integrating multi-objective constraints through a multi-objective optimization algorithm using the sorted list of technical achievements, and the specific steps are as follows:

[0030] Based on the sorted list of technical achievements, define technology maturity constraints, business value constraints, implementation cost constraints, and policy compliance constraints, output a multi-objective constraint set, and dynamically adjust the weights through a sliding window mechanism;

[0031] Based on the multi-objective constraint set and dynamic weight vector, the improved NSGA-III algorithm is used to select the first M items from the sorted list of technical achievements as the initial population, and a reference point set is uniformly generated in the four-dimensional target space.

[0032] Using simulated binary crossover and polynomial mutation to perform crossover and mutation operations, we generate an optimized population and perform non-dominated sorting, constructing a three-dimensional decision space. Through feature contribution analysis and visualization of key influencing factors, we generate a set of optimal solutions.

[0033] The set of preferred solutions is optimized through the internal and external double-loop optimization mechanism to generate an adaptive solution.

[0034] In a second aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the enterprise scientific and technological achievement adaptation method based on big data precise retrieval and query as described in the first aspect of the present invention is implemented.

[0035] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the enterprise scientific and technological achievement adaptation method based on big data precise retrieval and query as described in the first aspect of the present invention is implemented.

[0036] The beneficial effects of the present invention are as follows: multi-source heterogeneous data are mapped to a unified semantic space through a cross-modal embedding model, and a standardized multimodal feature matrix is ​​generated, which significantly improves the accuracy and efficiency of multimodal data fusion; combined with the construction and update of a dynamic knowledge graph, real-time and accurate retrieval of corporate scientific and technological achievements is achieved; using the sorted list of technical achievements, multi-objective constraints are integrated through a multi-objective optimization algorithm to generate an adaptation plan, which solves the problem that traditional methods are difficult to simultaneously meet multi-dimensional requirements such as technology, business, cost and policy, and provides efficient, accurate and scientific decision-making support for the management, retrieval and transformation of corporate technological achievements, and comprehensively improves the intelligence level and practical application effect of the adaptation of scientific and technological achievements. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 This is a flowchart of the enterprise scientific and technological achievement adaptation method based on big data precise retrieval and query in Example 1.

[0039] Figure 2 This is a flowchart of enterprise multi-source heterogeneous data preprocessing in Example 1.

[0040] Figure 3 Flowchart for cross-modal embedding model training and feature matrix generation in Example 1.

[0041] Figure 4 This is a flowchart for constructing and updating the dynamic knowledge graph in Example 1. DETAILED DESCRIPTION

[0042] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0043] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0044] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0045] Example 1, with reference to Figures 1 to 4 This embodiment provides a method for adapting enterprise scientific and technological achievements based on big data precise search and query, including the following steps:

[0046] S1, collects enterprise multi-source heterogeneous data, preprocesses it, and generates a standardized multimodal enterprise dataset.

[0047] Collect R&D project data, technical parameter tables, technical drawings and design documents to generate internal enterprise data, and collect industry data, supply chain data, industry reports and market analysis data to generate external enterprise data and obtain multi-source heterogeneous enterprise data.

[0048] Use interpolation to fill missing values, deduplicate unique identifier fields, filter invalid characters using regular expressions, and employ statistical methods to detect and process outliers.

[0049] It should be explained that missing value locations in enterprise multi-source heterogeneous data are identified, and appropriate interpolation methods (such as linear interpolation, polynomial interpolation, or time series interpolation) are selected based on the type of enterprise multi-source heterogeneous data. For numerical enterprise multi-source heterogeneous data, missing values ​​are calculated using interpolation formulas based on known values ​​of the enterprise multi-source heterogeneous data, and the calculated values ​​are filled in the missing locations;

[0050] Identify unique identification fields (such as IDs and numbers) in the enterprise's multi-source heterogeneous data, use deduplication algorithms (such as hash tables or sorting methods) to detect duplicate values, and retain the first record or specific records (such as the latest record) based on business rules (such as retaining the latest record, retaining the first record, or merging records) and delete subsequent duplicate records;

[0051] Identify the invalid characters that need to be filtered (such as special symbols, garbled characters, invisible characters, etc.) and write regular expressions to match these invalid characters, such as a regular expression to filter non-alphanumeric characters. Traverse text fields in the enterprise's multi-source heterogeneous data and use regular expressions to match and replace or delete invalid characters.

[0052] Based on the distribution characteristics of the enterprise's multi-source, heterogeneous data, select appropriate statistical methods (such as the Z-score (standardized score), IQR (interquartile range), and the 3σ rule) to detect outliers. For example, use the Z-score method to calculate the Z-score value for each data point and identify outliers based on Z-score thresholds (such as Z-score > 3 or Z-score < -3). Outliers are handled by deleting them, replacing them with the mean / median, or correcting them based on business rules.

[0053] Unify the units and time formats of structured data, convert unstructured data into a unified format, and encode categorical data to generate standardized multimodal enterprise datasets.

[0054] It should be explained how to identify unit and time fields in structured enterprise multi-source heterogeneous data and clearly define the need for unified formats (e.g., standardizing length units to "meters" and time formats to "year-month-day hour:minute:seconds"). Scripts or tools (e.g., Python's pandas library) should be used to traverse the enterprise's multi-source heterogeneous data, converting unit fields (e.g., converting "feet" to "meters") and formatting time fields (e.g., converting "day / month / year" to "year-month-day").

[0055] For unstructured enterprise multi-source heterogeneous data (such as text, images, and audio), define the target format (e.g., convert text to UTF-8 encoded plain text, images to JPEG format, and audio to MP3 format). Use appropriate tools (e.g., Python's Pillow library for image processing and pydub library for audio processing) to convert enterprise multi-source heterogeneous data, ensuring that all unstructured enterprise multi-source heterogeneous data conforms to a unified format standard.

[0056] Identify categorical fields (e.g., product category, technology type) within the enterprise's multi-source heterogeneous data and determine an encoding method (e.g., one-hot encoding, label encoding). Use encoding tools (e.g., Python's sklearn library) to transform categorical fields. For example, encode "Class A" and "Class B" in the "Product Category" column to [1, 0] and [0, 1], respectively. After encoding, verify the enterprise's multi-source heterogeneous data to ensure the numerical representation of the categorical fields is correct.

[0057] Integrate the processed structured and unstructured enterprise multi-source heterogeneous data, along with the coded and classified enterprise multi-source heterogeneous data, into a unified enterprise multi-source heterogeneous dataset. Use enterprise multi-source heterogeneous data integration tools (such as the Python pandas library) to link the different modalities of enterprise multi-source heterogeneous data based on unique identification fields to ensure the consistency and integrity of the enterprise multi-source heterogeneous data. Then, inspect the enterprise multi-source heterogeneous dataset to verify that all fields have been standardized, generating the final standardized multimodal enterprise dataset.

[0058] S2, maps the standardized multimodal enterprise dataset into a unified semantic space through a cross-modal embedding model to generate a standardized multimodal feature matrix.

[0059] It should be noted that cross-modal embedding models, such as CLIP, ViLBERT, LXMERT, and UNITER, aim to map data from different modalities (such as text, images, and audio) into a unified semantic space. The CLIP model is used here because it is pre-trained on a large-scale dataset of text-image pairs through contrastive learning and can efficiently learn semantic alignment for multimodal data.

[0060] Based on the historical standardized multimodal enterprise dataset, a multimodal training dataset is constructed. The contrastive loss function is used to minimize the embedding distance of positive sample pairs and maximize the embedding distance of negative sample pairs to train the CLIP model to learn the semantic alignment of multimodal data.

[0061] It should be noted that the historical standardized multimodal enterprise dataset is divided into multimodal sample pairs (e.g., text-image, text-table, etc.), and the positive and negative sample relationships are annotated for each sample pair (positive sample pairs are semantically related, and negative sample pairs are semantically unrelated). The CLIP model encoder is used to extract the embedding vectors of text and image (or other modalities), respectively, and the embedding distance (e.g., cosine similarity) of the positive sample pairs and the embedding distance of the negative sample pairs are calculated. The CLIP model parameters are optimized using a contrastive loss function (e.g., InfoNCE Loss) to minimize the embedding distance of positive sample pairs and maximize the embedding distance of negative sample pairs. During training, the CLIP model weights are iteratively updated using gradient descent, and the CLIP model performance is monitored using a validation set.

[0062] Based on the standardized multimodal enterprise dataset, the trained CLIP model is mapped to a unified semantic space, and the weight ratios of different modalities are dynamically adjusted to form a standardized multimodal feature matrix.

[0063] It should be explained that based on the standardized multimodal enterprise dataset, the trained CLIP model is used to extract the embedding vectors of modalities such as text, image, and table, and the embedding vectors of all modalities are mapped to a unified semantic space. According to business needs or data characteristics, an initial weight is assigned to each modality (such as text weight 0.5, image weight 0.3, table weight 0.2), and the weight ratio is optimized through a dynamic adjustment algorithm (such as a feedback mechanism based on modality importance or task performance). The embedding vectors of each modality are weighted and combined according to the adjusted weights to generate a standardized multimodal feature matrix.

[0064] S3, builds a knowledge graph in the enterprise technology field based on a standardized multimodal feature matrix.

[0065] Based on the enterprise's multi-source heterogeneous data, technical entities, enterprise entities and external entities are extracted, and based on the explicit associations between the enterprise's multi-source heterogeneous data, three types of relationships, namely technology, enterprise and external, are defined.

[0066] The document should explain the use of named entity recognition (NER) tools (such as SpaCy or Stanford NLP) to extract technical entities (such as patents and technical terms), corporate entities (such as company names and departments), and external entities (such as industry standards, policies and regulations) from text data based on multi-source heterogeneous enterprise data. The document should also explain the use of relationship extraction algorithms (such as rule-based methods or deep learning models) to identify explicit associations (such as "Company A developed Technology B" or "Policy C influenced Technology D") from multi-source heterogeneous enterprise data. The document should also explain the definition of three types of relationships based on these associations: technical relationships (such as technology dependency and technology citation), corporate relationships (such as corporate collaboration and department affiliation), and external relationships (such as policy influence and industry standard citation).

[0067] It should also be noted that SpaCy is a high-performance, easy-to-use open source NLP library suitable for industrial applications; Stanford NLP is an NLP toolkit developed by Stanford University that provides high-quality models.

[0068] The same entity in multi-source heterogeneous data sources of different enterprises is merged into a unique node through the entity matching algorithm, and the same relationship is standardized through the relationship mapping rules.

[0069] It should be explained that entity matching algorithms (such as rule-based methods, machine learning models or pre-trained entity linking tools) are used to match entities in multi-source heterogeneous data sources of different enterprises, and whether they are the same entity is determined by comparing the attributes of the entities (such as name, description, ID) and contextual information (such as associated entities, events), and the matching entities are merged into a unique node (such as merging "Company A" and "A Corp" into "Company A"), defining relationship mapping rules (such as unifying "Development" and "R&D" into "Development"), traversing all relationship data, standardizing the same relationship according to the relationship mapping rules (such as mapping "R&D" to "Development"), and updating the relationship names in the relationship data.

[0070] Neo4j is used to store entities and relationships as knowledge graph nodes and edges, and the entity semantic vectors and cross-modal association information in the standardized multimodal feature matrix are used as node attributes and relationship weights to obtain the enterprise technology field knowledge graph.

[0071] It should be explained how to create a node and edge data structure in Neo4j, use the extracted entities as nodes (such as "Company A" and "Technology B"), and the defined relationships as edges (such as "Development"). Use Cypher statements (such as CREATE and MERGE9) to batch import entities and relationships into Neo4j, ensure the uniqueness of nodes and edges, use the entity semantic vectors in the standardized multimodal feature matrix as node attributes, and use cross-modal association information as relationship weights. Verify the integrity and consistency of the knowledge graph, ensure that nodes, edges, and their attributes are correctly stored, and output the final knowledge graph.

[0072] It should also be explained that verifying the integrity and consistency of the knowledge graph specifically involves: checking whether the number of nodes and edges meets expectations, ensuring that all entities and relationships have been correctly imported, verifying the integrity of node attributes (such as checking whether the attributes exist) and the correctness of relationship weights (such as checking whether the weight values ​​are reasonable) through Cypher queries, and using graph traversal algorithms (such as breadth-first search or depth-first search) to check the connectivity of the knowledge graph to ensure that there are no isolated nodes or broken relationships.

[0073] S4 uses the Apache Flink streaming processing framework to receive incremental technical data in real time, dynamically update graph node relationships, remove expired nodes through a pruning algorithm, and output a dynamic knowledge graph with timestamps.

[0074] Through Apache Flink, we access incremental technical data streams in real time, parse and extract entities, relationships, and timestamps, and handle out-of-order events based on event time distribution and watermarks. We use dynamic update strategies to aggregate operations within time windows and write them in batches to Neo4j through asynchronous I / O. At the same time, we prune expired nodes in the enterprise technology field knowledge graph based on node activity tags and business rules, execute Cypher deletion operations regularly, and ultimately output a dynamic knowledge graph with timestamps.

[0075] The details are as follows:

[0076] Use Apache Flink's Kafka connector to subscribe to incremental data streams (such as new patents, technical parameter updates, and policy changes) in real time, configure Kafka topics and consumer groups, and output incremental technical data streams.

[0077] Use Flink's DataStream API to parse incremental technical data streams one by one, verify field integrity (for example, patent IDs and technical terms cannot be null), extract key fields (entity IDs, event types, timestamps, and relationships), filter out invalid data that is missing necessary fields or has incorrect formatting, and output a structured incremental data stream.

[0078] Assign timestamps based on the event time fields in the structured incremental data stream and generate watermarks. This allows for a fixed delay (e.g., 5 minutes) to process out-of-order events. Configure watermark policies (e.g., BoundedOutOfOrderness) to ensure that the structured incremental data stream is correctly processed within a specified time window and output an incremental data stream with timestamps and watermarks.

[0079] BoundedOutOfOrderness is a watermark generation strategy that allows data to arrive out of order within a specified time range;

[0080] Dynamically generate Cypher operation instructions based on the event types in the incremental data stream with timestamps and watermarks: If the incremental data stream contains new entities (such as a new patent ID), a CREATE operation is generated to create the node and initialize the attributes; if the incremental data stream contains attribute changes (such as technical term expansion), a SET operation is generated to update the node attributes; if the incremental data stream contains new associations (such as patents citing new standards), a MERGE operation is generated to create or update the relationship, record the association weight, and output a Cypher operation instruction stream (a set of add, delete, and modify operations);

[0081] Aggregate multiple update operations for the same entity in a Cypher operation instruction stream based on a sliding window (e.g., a 10-minute window with a 5-minute sliding step). Group and merge similar operations by entity ID (e.g., merge multiple attribute updates into a single SET operation) to reduce write frequency. Output the aggregated batch Cypher operation instruction stream.

[0082] Call the Neo4j driver through Flink's asynchronous I / O interface, execute Cypher operations in batches, use the UNWIND statement to optimize batch write performance, configure asynchronous request timeouts and maximum concurrency, and use Neo4j's batch transaction interface to submit operations. Output the real-time updated enterprise technology domain knowledge graph nodes and relationships (including the latest attributes and timestamps).

[0083] Add a last_updated field to each node in the real-time updated enterprise technology domain knowledge graph nodes and relationships (including the latest attributes and timestamps), record the last updated timestamp, define business expiration rules (for example, if a technology node has not been updated for more than 5 years, it will be marked as expired), automatically write the current timestamp when the entity is updated, regularly scan nodes and add expiration tags (such as Expired), and output a set of nodes marked as expired;

[0084] Expired is a label or status indicating that a node or data has expired or is no longer used;

[0085] Based on the time decay function (such as ,in, is the node activity score, (The time difference represents the interval between the current time and the last time the node was active) Calculate the node activity score in the set of nodes marked as expired. Nodes below the elimination threshold are considered expired. At the same time, specific types of nodes (such as abolished industry standards and invalid policies) are directly deleted. Cypher statements are executed to match expired nodes, and nodes and associated relationships are batch deleted. The knowledge graph of the enterprise technology field after the expired nodes are removed is output;

[0086] Use Flink's ProcessFunction to trigger pruning tasks regularly (for example, at dawn every day) to avoid execution during peak hours of online services. Register a timer to call Neo4j's Cypher interface to perform deletion operations, record pruning logs, and output a dynamic knowledge graph with timestamps.

[0087] ProcessFunction is a low-level API provided for implementing custom stream processing logic.

[0088] S5, receiving a user query request, extracting a query vector using the cross-modal embedding model, and retrieving a Top-N candidate node list through a dynamic knowledge graph.

[0089] Use the Flask framework to build an API interface, define the JSON format of query requests, receive user query requests, parse user query requests through natural language processing tools, extract query keywords and verify the legitimacy of parameters.

[0090] It should be explained that the API interface is created using the Flask framework, the JSON format of the query request is defined, and routes are set up to receive user query requests. User query requests are parsed using natural language processing tools (such as spaCy or NLTK), query keywords (such as "Company A" and "Technology") are extracted, and parameter validity is verified (for example, whether the query field is empty and whether the limit is an integer and within a reasonable range);

[0091] query is a field used to pass the query content entered by the user (such as search keywords, filter conditions, etc.);

[0092] limit is a field that specifies the maximum number of results to return.

[0093] The trained CLIP model is used to map query keywords to a unified semantic space, and a text encoder is used to output the query vector.

[0094] It should be explained that the extracted query keywords (such as "Company A's technology") are input into the text encoder of the trained CLIP model (such as CLIP's text encoder), the query keywords are converted into text embedding vectors, the query keywords are segmented and standardized (such as converting to lowercase and removing stop words) through the preprocessing module of the CLIP model, ensuring that the input format meets the requirements of the CLIP model, calling the forward propagation method of the text encoder, calculating the semantic vector of the query keywords, and outputting the query vector (such as a fixed-length floating-point array).

[0095] Based on the dynamic knowledge graph, FAISS is used to index all node vectors, and the search method is used to retrieve the Top-N candidate nodes. The semantic similarity scores between the query vector and some node vectors are calculated through the approximate nearest neighbor algorithm, and the Top-N candidate node list is output.

[0096] It should be explained that the semantic vectors of all nodes in the dynamic knowledge graph (such as the embedding vectors generated by the CLIP model) are loaded into FAISS, the vector index is built using an index structure such as IndexFlatL2 or IndexIVFFlat, the search method of FAISS is called, the query vector is used as input, and the search parameters (such as Retrieve the top-N nodes), calculate the semantic similarity score (such as Euclidean distance or cosine similarity) between the query vector and the node vector using an approximate nearest neighbor algorithm (ANN), sort the candidate nodes in descending order (from high to low) based on the similarity score, and output a list of the top-N candidate nodes (such as in JSON format, including node IDs and similarity scores);

[0097] Calculate the semantic similarity score between the query vector and the node vector. The expression is:

[0098] ;

[0099] in, The cosine value of the query vector and some node vectors is in the range of [-1, 1], is the query vector, is the node vector, is the query vector and node vector The angle between

[0100] FAISS is an efficient similarity search library developed by Facebook AI Research, specifically designed to quickly find the most similar vectors in large-scale vector data;

[0101] search is a method used to find the vector most similar to the query vector in the constructed vector index;

[0102] IndexFlatL2 (Flat Index Based on L2 Distance) is an index structure in FAISS that uses L2 distance (Euclidean distance) to calculate the similarity between vectors;

[0103] IndexIVFFlat (Flat Index Based on Inverted File Index) is an index structure in FAISS that combines an inverted file index and a flat index to accelerate searches on large-scale vector data.

[0104] S6, calculates the comprehensive similarity score based on the graph relationship weights and outputs a sorted list of technical achievements.

[0105] Based on the list of Top-N candidate nodes, query the relationship weights of the Top-N candidate nodes from the knowledge graph, output the Top-N candidate nodes and relationship weights, adjust the semantic similarity score according to the relationship weights, obtain the comprehensive similarity score, and sort it in descending order, intercept the Top-K results, and output the sorted list of technical achievements.

[0106] It should be noted that based on the Top-N candidate node list, each candidate node is traversed, and the relationship weight of each candidate node is retrieved from the knowledge graph using Cypher query. The retrieved relationship weight is combined with the semantic similarity score, and the semantic similarity score is adjusted according to the relationship weight to obtain a comprehensive similarity score, which is expressed as:

[0107] ;

[0108] in, is the comprehensive similarity score, is the semantic similarity score between the query vector and some node vectors, is the relationship weight coefficient, is the relationship weight of the Top-N candidate nodes;

[0109] The comprehensive similarity scores of the candidate nodes are stored in a list, and sorted in descending order using a sorting algorithm (such as Python's sorted() function). Before value truncation Nodes, format the information of Top-K nodes (such as node ID, comprehensive score, technical description) into a list of technical achievements.

[0110] S7, using the sorted list of technical achievements, integrates multi-objective constraints through a multi-objective optimization algorithm to generate an adaptation solution.

[0111] Based on the sorted list of technological achievements, technology maturity constraints, commercial value constraints, implementation cost constraints, and policy compliance constraints are defined, a multi-objective constraint set is output, and the weights are dynamically adjusted through a sliding window mechanism.

[0112] It should be stated that the maturity-related indicators of each technology (such as the technology development stage, number of patents, and degree of technology verification) are extracted from the sorted list of technological achievements, and these indicators are quantified using a scoring standard (such as 1-10 points). For example, a technology in the experimental stage is scored as 2, a technology in the pilot stage is scored as 5, and a technology in the commercialization stage is scored as 8. The maturity score of each technology is stored as a technology maturity constraint;

[0113] Extract the commercial value indicators (such as expected revenue, market share, and return on investment) of each technology from the sorted list of technological achievements. Calculate these indicators using quantitative methods (such as amounts or percentages). For example, expected revenue of less than 1 million yuan is scored as 1, 1 million to 5 million yuan is scored as 3, and 5 million yuan or more is scored as 5. Store the commercial value score of each technology as a commercial value constraint.

[0114] Extract implementation cost indicators (such as R&D budget, equipment investment, and labor costs) for each technology from the sorted list of technological achievements. Calculate these indicators using quantitative methods (such as monetary amounts or cost scores). For example, implementation costs below 500,000 yuan are scored as 5, between 500,000 and 1,000,000 yuan are scored as 3, and those above 1,000,000 yuan are scored as 1. Store the implementation cost score for each technology as an implementation cost constraint.

[0115] Extract each technology's policy compliance indicators (e.g., regulatory compliance, industry standards, environmental requirements) from the sorted list of technical achievements and evaluate them using quantitative methods (e.g., compliance scores or Boolean values). For example, full compliance is scored as 5, partial compliance is scored as 3, and non-compliance is scored as 1. Store each technology's policy compliance score as a policy compliance constraint.

[0116] Integrate technology maturity constraints, business value constraints, implementation cost constraints, and policy compliance constraints into a multi-objective constraint set (such as in JSON format) to ensure that the constraint values ​​of each technological achievement are complete and can be used by subsequent optimization algorithms. Dynamically adjust weights through a sliding window mechanism (such as a fixed time window or an event-triggered window): within the window, calculate the weight changes of each constraint based on real-time data (such as market feedback and technology updates) (such as an increase in business value weight and a decrease in implementation cost weight), update the weight vector, and output the adjusted dynamic weight vector and multi-objective constraint set.

[0117] Based on the multi-objective constraint set and dynamic weight vector, the improved NSGA-III algorithm is used to select the first M items from the sorted list of technical achievements as the initial population, and generate a reference point set uniformly in the 4-dimensional target space.

[0118] It should be noted that based on the multi-objective constraint set and dynamic weight vector, the top Item as the initial population (such as ), ensure that the technical maturity, commercial value, implementation cost, and policy compliance constraints of each technological achievement are clearly defined. In the four-dimensional target space (technical maturity, commercial value, implementation cost, and policy compliance), use a uniform distribution method (such as the Das-Dennis method) to generate a set of reference points. Ensure that the reference points are evenly distributed in the target space. Map the individuals of the initial population to the target space, and calculate the distance between each individual and the reference point.

[0119] Through the non-dominated sorting and reference point association mechanism of the improved NSGA-III algorithm, individuals in the population are non-dominated sorted and classified according to the Pareto frontier level, ensuring that individuals with higher frontier levels are selected first. Individual characteristics (such as technical indicators) are standardized and dimensionally reduced, and individual characteristics are mapped to coordinate points in the target space (such as [0.7, 0.3]). The Euclidean distance between the coordinate point in the target space and all predefined reference points is calculated, and this individual is assigned to the closest reference point (for example, reference point B is closer to 0.28 than reference point A, which is 0.36, so B is selected). Individuals are selected based on the crowding of the reference point (such as the number of associated individuals), and individuals associated with reference points with lower crowding are preferentially selected as the optimal individuals to ensure population diversity. The selected optimal individuals are added to the next generation population, and the optimal individuals are selected from them to enter the next generation population.

[0120] It should also be noted that the improved NSGA-III algorithm, based on the standard NSGA-III, significantly improves the diversity, convergence and computational efficiency of the algorithm in high-dimensional multi-objective optimization problems by optimizing reference point generation (such as adopting a dynamic reference point generation method to adaptively adjust the reference point position according to the population distribution), crossover and mutation strategies (such as combining simulated binary crossover SBX and polynomial mutation PM, or designing customized operators), constraint processing mechanisms (such as introducing an adaptive penalty function to dynamically adjust constraint priorities), population update strategies (such as adding an elite retention mechanism and local search technology to accelerate convergence) and parallel computing (such as using GPU acceleration or distributed computing framework to improve operation efficiency). Its feasibility and practicality have been proven through theoretical analysis and experimental verification.

[0121] Simulated binary crossover and polynomial mutation are used to perform crossover and mutation operations to generate an optimized population and perform non-dominated sorting. A three-dimensional decision space is constructed, and a set of preferred solutions is generated through feature contribution analysis and visualization of key influencing factors.

[0122] It should be explained that a simulated binary crossover (SBX) operation is performed on individuals in the initial population, parent individuals are randomly selected, offspring individuals are generated based on the crossover probability and distribution index, polynomial mutation (PM) is performed on the offspring individuals, individual gene values ​​are randomly adjusted based on the mutation probability and distribution index to generate an optimized population, the optimized population is non-dominated sorted, individuals are classified according to Pareto frontier levels, and visualized in a three-dimensional decision space (such as technology maturity, commercial value, and implementation cost), the characteristic values ​​of each technological achievement (such as technology maturity, commercial value, and implementation cost) are extracted, and the contribution of each technological achievement's characteristic value to the overall score is calculated (such as by weight or score ratio). A feature contribution graph is drawn using visualization tools (such as Matplotlib or Seaborn), and technological achievements with significant feature contributions that meet the constraints are screened out. These are then sorted by overall score to generate a set of preferred solutions;

[0123] Matplotlib is a Python drawing library for creating various static, dynamic, and interactive charts;

[0124] Seaborn is a high-level Python data visualization library based on Matplotlib, focusing on drawing statistical charts.

[0125] The set of preferred solutions is optimized through the internal and external double-loop optimization mechanism to generate an adaptive solution.

[0126] It should be explained that in the outer loop, based on the set of preferred solutions, a global optimization goal (e.g., maximizing commercial value, minimizing implementation costs) is set, and solution parameters (e.g., technology maturity, implementation budget) are adjusted through a global search algorithm (e.g., genetic algorithm or particle swarm optimization) to generate a preliminary optimization solution. Then, through the inner loop, each preliminary optimization solution is locally optimized, and a local search algorithm (e.g., gradient descent or simulated annealing) is used to fine-tune the technical implementation decision variables (e.g., technical implementation details, resource allocation), and verify the feasibility of the solution under constraints (e.g., policy compliance, technical feasibility). The results of the inner and outer loop optimizations are integrated to generate an adapted solution.

[0127] It should also be noted that the internal and external dual circulation optimization mechanism:

[0128] Internal circulation mechanism:

[0129] Definition: The internal loop mechanism is a high-frequency, real-time optimization process that rapidly adjusts model parameters or local variables by continuously receiving dynamic data streams (such as market changes and technological status updates). Its core is to achieve rapid iteration through incremental learning or online learning, for example, updating a business value prediction model every 30 minutes.

[0130] Application areas: In existing technologies, the internal loop mechanism is widely used in real-time recommendation systems (such as dynamic pricing on e-commerce platforms), industrial Internet of Things (such as equipment status monitoring and predictive maintenance), and financial high-frequency trading (such as real-time stock price prediction).

[0131] External circulation mechanism:

[0132] Definition: The outer loop mechanism is a low-frequency, global optimization process that focuses on long-term policy adjustments and system-level parameter optimization. For example, a multi-objective weight allocation strategy is periodically updated through reinforcement learning (such as the PPO algorithm), usually in batches on a daily or weekly basis.

[0133] Application areas: The external loop mechanism has mature applications in automated control systems (such as robot path planning), resource scheduling (such as cloud computing task allocation), and long-term risk modeling (such as insurance actuarial science).

[0134] The combination of the internal and external dual-loop optimization mechanism of the present invention is as follows:

[0135] Synergy between data and strategy:

[0136] Internal loop: Real-time processing of the timeliness of technological achievements;

[0137] Outer loop: Based on the real-time data accumulated in the inner loop, streaming feedback (user adoption data) drives reinforcement learning (PPO algorithm) and dynamically adjusts the weights of multiple objectives;

[0138] Technology integration and innovation:

[0139] Combine the inner-loop streaming computing framework (Apache Flink) with the outer-loop offline reinforcement learning engine, and achieve dual-cycle data synchronization through a shared state storage (such as Redis or HBase);

[0140] The "fast decision-making" of the inner loop focuses on high-frequency data updates, and the "slow strategy" of the outer loop focuses on low-frequency strategy optimization, which complement each other. The two work together through sharing state information and feedback mechanisms to solve the problem of balancing real-time and globality in a single loop.

[0141] This embodiment also provides a computer device, which is suitable for the enterprise scientific and technological achievements adaptation method based on big data precise retrieval and query, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the enterprise scientific and technological achievements adaptation method based on big data precise retrieval and query proposed in the above embodiment.

[0142] The computer device may be a terminal, comprising a processor, memory, a communication interface, a display, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage media. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication. Wireless communication may be achieved via Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. The display of the computer device may be a liquid crystal display or an electronic ink display. The input device may be a touchscreen overlay on the display, buttons, a trackball, or a touchpad on the computer device housing, or an external keyboard, touchpad, or mouse.

[0143] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the enterprise scientific and technological achievement adaptation method based on big data precise retrieval and query as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0144] In summary, the present invention achieves significant improvements in the accuracy and efficiency of multimodal data fusion by: mapping multi-source heterogeneous data into a unified semantic space through a cross-modal embedding model, generating a standardized multimodal feature matrix; combining the construction and update of a dynamic knowledge graph to achieve real-time and accurate retrieval of corporate scientific and technological achievements; utilizing a sorted list of technological achievements, integrating multi-objective constraints through a multi-objective optimization algorithm to generate an adaptation solution, solving the problem that traditional methods are difficult to simultaneously meet multi-dimensional requirements such as technology, business, cost, and policy, providing efficient, accurate, and scientific decision-making support for the management, retrieval, and transformation of corporate technological achievements, and comprehensively improving the intelligence level and practical application effect of the adaptation of scientific and technological achievements.

[0145] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for adapting enterprise scientific and technological achievements based on accurate retrieval and query of big data, characterized by: include: Collect enterprise multi-source heterogeneous data, perform pre-processing, and generate standardized multimodal enterprise data sets; The standardized multimodal enterprise dataset is mapped to a unified semantic space through a cross-modal embedding model to generate a standardized multimodal feature matrix. The specific steps are as follows: Based on a historical standardized multimodal enterprise dataset, a multimodal training dataset is constructed. The contrastive loss function is used to minimize the embedding distance of positive sample pairs and maximize the embedding distance of negative sample pairs to train the CLIP model to learn the semantic alignment of multimodal data. Based on a standardized multimodal enterprise dataset, the trained CLIP model is mapped to a unified semantic space, and the weight ratios of different modalities are dynamically adjusted to form a standardized multimodal feature matrix. Based on a standardized multimodal feature matrix, we build a knowledge graph for the enterprise's technical field. We use the Apache Flink streaming framework to receive incremental technical data in real time, dynamically update graph node relationships, and remove expired nodes through a pruning algorithm to output a dynamic knowledge graph with timestamps. The specific steps of outputting a dynamic knowledge graph with a timestamp are as follows: Through Apache Flink, we access incremental technical data streams in real time, parse and extract entities, relationships, and timestamps, and handle out-of-order events based on event time distribution and watermarks. We use dynamic update strategies to aggregate operations within a time window and write them in batches to Neo4j via asynchronous I / O. We also prune expired nodes in the enterprise technology domain knowledge graph based on node activity tags and business rules, execute Cypher delete operations regularly, and ultimately output a dynamic knowledge graph with timestamps. Receive user query requests, extract query vectors using the cross-modal embedding model, retrieve the Top-N candidate node list through the dynamic knowledge graph, calculate the comprehensive similarity score based on the graph relationship weights, and output a ranked list of technical achievements; Using the sorted list of technical achievements, we integrate multi-objective constraints through a multi-objective optimization algorithm to generate an adaptation solution. The specific steps are as follows: Based on the sorted list of technical achievements, define technology maturity constraints, business value constraints, implementation cost constraints, and policy compliance constraints, output a multi-objective constraint set, and dynamically adjust the weights through a sliding window mechanism; Based on the multi-objective constraint set and dynamic weight vector, the improved NSGA-III algorithm is used to select the first M items from the sorted list of technical achievements as the initial population, and a reference point set is uniformly generated in the four-dimensional target space. Using simulated binary crossover and polynomial mutation to perform crossover and mutation operations, we generate an optimized population and perform non-dominated sorting, constructing a three-dimensional decision space. Through feature contribution analysis and visualization of key influencing factors, we generate a set of optimal solutions. The set of preferred solutions is optimized through the internal and external double-loop optimization mechanism to generate an adaptive solution.

2. The enterprise scientific and technological achievement adaptation method based on big data precise search and query according to claim 1 is characterized by: The specific steps of collecting enterprise multi-source heterogeneous data and preprocessing it to generate a standardized multimodal enterprise data set are as follows: Collect R&D project data, technical parameter tables, technical drawings and design documents to generate internal enterprise data, and collect industry data, supply chain data, industry reports and market analysis data to generate external enterprise data and obtain multi-source heterogeneous enterprise data; Use interpolation to fill missing values, remove duplicates from unique identifier fields, use regular expressions to filter invalid characters, and use statistical methods to detect and handle outliers; Unify the units and time formats of structured data, convert unstructured data into a unified format, and encode categorical data to generate standardized multimodal enterprise datasets.

3. The enterprise scientific and technological achievement adaptation method based on big data precise search and query according to claim 1 is characterized by: The above method is based on the standardized multimodal feature matrix to construct a knowledge graph in the enterprise technology field, as follows: Based on the enterprise's multi-source heterogeneous data, technical entities, enterprise entities, and external entities are extracted, and based on the explicit associations between the enterprise's multi-source heterogeneous data, three types of relationships are defined: technology, enterprise, and external; Merge the same entity in heterogeneous data sources from different enterprises into a unique node through entity matching algorithms, and standardize the same relationship through relationship mapping rules; Neo4j is used to store entities and relationships as knowledge graph nodes and edges, and the entity semantic vectors and cross-modal association information in the standardized multimodal feature matrix are used as node attributes and relationship weights to obtain the enterprise technology field knowledge graph.

4. The enterprise scientific and technological achievement adaptation method based on big data precise search and query according to claim 1 is characterized by: The steps of receiving a user query request, extracting a query vector using the cross-modal embedding model, and retrieving a list of top-N candidate nodes through a dynamic knowledge graph are as follows: Use the Flask framework to build an API interface, define the JSON format of query requests, receive user query requests, parse user query requests through natural language processing tools, extract query keywords and verify the legitimacy of parameters; The trained CLIP model maps query keywords to a unified semantic space and uses a text encoder to output a query vector. Based on the dynamic knowledge graph, FAISS is used to index all node vectors, and the search method is used to retrieve the Top-N candidate nodes. The semantic similarity scores between the query vector and some node vectors are calculated through the approximate nearest neighbor algorithm, and the Top-N candidate node list is output.

5. The enterprise scientific and technological achievement adaptation method based on big data precise search and query according to claim 1 is characterized by: Based on the list of Top-N candidate nodes, the relationship weights of the Top-N candidate nodes are queried from the dynamic knowledge graph, and the Top-N candidate nodes and relationship weights are output. The semantic similarity scores are adjusted according to the relationship weights to obtain a comprehensive similarity score, which is sorted in descending order. The Top-K results are intercepted and a sorted list of technical achievements is output.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the enterprise scientific and technological achievement adaptation method based on big data precise retrieval and query are implemented as described in any one of claims 1 to 5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the enterprise scientific and technological achievement adaptation method based on big data precise retrieval and query are implemented as described in any one of claims 1 to 5.

Images