Project research and development data key information processing method and device

Abstract

The embodiment of the invention provides a project research and development data key information processing method and device. A research and development content recognition system and a self-adaptive research and development knowledge graph are constructed. Unified processing and time sequence alignment of text, voice and image contents are realized through multi-modal information decomposition and fusion. Semantic completion and error correction are carried out based on research and development of a semantic analysis model, a knowledge graph structure is dynamically constructed and optimized, and a structured document with a traceability relation is generated. The system adopts a deep neural network model to extract research and development key information, constructs a multi-level document framework, and realizes intelligent conversion from research and development data to a project application document. According to the method, the defects of the traditional technology in the aspects of multi-modal information processing and knowledge structure optimization are effectively overcome, and the research and development data management and project declaration efficiency is remarkably improved.

Description

Technical Field The present application relates to the field of data processing, and specifically to a method and device for processing key information of project R & D materials. Background Art There are obvious deficiencies in the existing R & D material processing technologies. Traditional methods often focus on single text processing and lack the ability to comprehensively process multi-modal information. They are unable to effectively integrate and utilize voice records and image materials during the R & D process, resulting in insufficient information utilization and weak relevance.

[0001] In addition, there are bottlenecks in knowledge graph construction and dynamic optimization in the existing technologies. Most systems adopt static knowledge structures and lack the ability of adaptive adjustment, making it difficult to reflect the dynamic correlation relationships between R & D knowledge. The document processing process lacks intelligent semantic understanding and error correction mechanisms, affecting the quality and accuracy of documents.

[0002] There are technical shortcomings in the construction of multi-level document frameworks and consistency verification in existing systems. There is a lack of accurate assessment of the importance of R & D information, and it is impossible to achieve reasonable stratification and intelligent organization of document content. Solving these problems is of great significance for improving the efficiency of R & D material management and the quality of project declarations. Summary of the Invention In view of the problems in the existing technologies, the present application provides a method and device for processing key information of project R & D materials, which can effectively solve the deficiencies of traditional technologies in multi-modal information processing and knowledge structure optimization, and significantly improve the efficiency of R & D material management and project declaration.

[0003] To solve at least one of the above problems, the present application provides the following technical solutions: In a first aspect, the present application provides a method for processing key information of project R & D materials, including: Importing enterprise R & D materials into a R & D content recognition system, decomposing the R & D materials into multi-modal information through the R & D content recognition system, extracting text content, voice content and image content, performing semantic segmentation on the text content, separating speakers and performing semantic recognition on the voice content, performing scene understanding and text recognition on the image content, aligning the text content, voice content and image content in the time series dimension, constructing a multi-modal information fusion matrix, and generating a semantic association vector based on the multi-modal information fusion matrix; Construct an adaptive R & D knowledge graph, input the semantic association vector into the R & D semantic analysis model. The R & D semantic analysis model performs semantic completion and error correction on the semantic association vector based on the context relationship, generates a normalized semantic unit, constructs the normalized semantic unit into a knowledge graph node, calculates the node weight based on the association degree between the knowledge graph nodes, dynamically adjusts the knowledge graph structure according to the node weight, and optimizes the topological structure of the adaptive R & D knowledge graph through feedback iteration to generate a structured R & D document with traceability relationships; Input the structured R & D document into the R & D document understanding model trained based on a deep neural network. The R & D document understanding model extracts key R & D information according to the topological structure of the adaptive R & D knowledge graph, constructs a multi-level document framework based on the weight distribution of the key information, fills the information in the structured R & D document into the multi-level document framework according to the information weight, and performs document consistency verification to generate a project application document.

[0004] Further, it also includes: classifying and storing R & D materials in digital format into a preset file directory according to the file format type, reading the file content in the preset file directory based on a file parser, parsing the file content into an initial data stream according to the data structure, dividing the initial data stream into a text data stream, an audio data stream, and an image data stream through the R & D content recognition system, performing text encoding conversion and format standardization processing on the text data stream, performing audio noise reduction and waveform normalization processing on the audio data stream, and performing image denoising and size standardization processing on the image data stream; Based on a semantic segmentation model, the text data stream is segmented into independent semantic segments, the voiceprint features of the audio data stream are extracted and a speaker voiceprint library is constructed, the audio data stream is transcribed into text content using a speech recognition model, scene classification and object detection are performed on the image data stream based on a convolutional neural network model, the text contained in the image is converted into text format using an optical character recognition model, and the semantic segments, transcribed text, and recognized text are combined to construct a unified text corpus.

[0005] Further, it also includes: performing temporal sorting on the text content, speech content, and image content based on timestamps, calculating the time interval between adjacent contents, extracting the temporal features of each modality content using a bidirectional long short-term memory network, constructing a temporal feature vector, performing normalization processing on the temporal feature vector, calculating the temporal alignment degree between different modality contents based on the dynamic time warping algorithm, and marking the contents with a temporal alignment degree higher than a preset threshold as associated segments; Map the associated fragments to a feature space to construct a feature matrix, use the attention mechanism to calculate the association weights between different modality features, perform weighted fusion on the feature matrix based on the association weights to obtain a multi-modal information fusion matrix, and use a tensor decomposition method to perform dimensionality reduction processing on the multi-modal information fusion matrix, and extract the main feature components to construct a semantic association vector.

[0006] Further, it also includes: constructing a research and development domain ontology library, using the concept nodes, relationship types, and attribute constraints in the ontology library as the basic framework of the knowledge graph, mapping the semantic association vector to the graph space based on the knowledge graph construction rules, using a named entity recognition model to identify entity types from the semantic association vector, using a relationship extraction model to extract the semantic relationships between entities, and integrating the identified entities and relationships into an initial knowledge graph; Use a bidirectional Transformer model to perform context encoding on the semantic association vector, calculate the semantic relevance between tokens based on the attention scores, complete the missing semantic information in the semantic association vector, construct a semantic error correction model to correct ambiguous semantic expressions, and construct a normalized semantic unit according to the preset rules for the completed and corrected semantic information.

[0007] Further, it also includes: mapping the normalized semantic unit to a knowledge graph node according to the node construction rules, extracting feature vectors for the knowledge graph node, using a graph attention network to calculate the semantic similarity between nodes, constructing a node adjacency matrix, calculating the node importance based on the node degree centrality and the eigenvector centrality, combining the node importance and the semantic similarity to obtain a node weight value, and using the node weight value as the edge weight of the knowledge graph; Construct a graph optimization objective function based on the edge weights, use the gradient descent method to iteratively optimize the topological structure of the knowledge graph, evaluate the connectivity and coverage of the knowledge graph in each iteration, obtain a stable knowledge graph structure when the optimization objective function converges, hierarchically organize the node information based on the knowledge graph structure, and convert the organized node information into a structured research and development document according to the preset template.

[0008] Further, it also includes: constructing a multi-layer perceptron neural network as the basic architecture of the document understanding model, converting the structured research and development document into a word vector sequence and inputting it into the neural network, setting a convolutional layer in the neural network to extract local features, setting a pooling layer to reduce the feature dimension, setting a fully connected layer to fuse the feature information, using the batch normalization method to standardize the feature data, and introducing a non-linear transformation through an activation function to obtain a deep feature representation of the document; Convert the topological structure of the adaptive R & D knowledge graph into a graph structure feature vector, use a graph convolutional network to extract features from the graph structure feature vector, fuse the extracted features with the deep document features, calculate the relevance between the fused features and the R & D key information based on the attention mechanism, and select the features with a relevance higher than the preset threshold as the R & D key information.

[0009] Further, it also includes: performing hierarchical clustering analysis on the R & D key information, calculating the semantic distance between the information, constructing a hierarchical clustering tree based on the semantic distance, mapping the hierarchical structure of the hierarchical clustering tree to the hierarchical structure of the document framework, setting the information display priority according to the weight value of the information in each level, converting the hierarchical structure into a multi-level document framework based on the preset document organization rules, and setting information filling position identifiers in the multi-level document framework; Fill the information in the structured R & D document into the corresponding information filling positions in descending order of the weight value, use a text similarity algorithm to detect the information redundancy in the filled document, merge and streamline the redundant information based on the preset rules, construct a document verification model to verify the structural integrity and content consistency of the document, optimize and adjust the document according to the verification results, and output the adjusted document as a project application document.

[0010] In a second aspect, the present application provides a device for processing key information of project R & D materials, including: A project material extraction module, configured to import enterprise R & D materials into a R & D content recognition system, perform multi-modal information decomposition on the R & D materials through the R & D content recognition system, extract text content, speech content, and image content, perform semantic segmentation on the text content, perform speaker separation and semantic recognition on the speech content, perform scene understanding and text recognition on the image content, align the text content, speech content, and image content in the time sequence dimension, construct a multi-modal information fusion matrix, and generate a semantic association vector based on the multi-modal information fusion matrix; A knowledge graph construction module, configured to construct an adaptive R & D knowledge graph, input the semantic association vector into a R & D semantic analysis model, the R & D semantic analysis model performs semantic completion and error correction on the semantic association vector based on the context relationship, generates a normalized semantic unit, constructs the normalized semantic unit into a knowledge graph node, calculates the node weight based on the association degree between the knowledge graph nodes, dynamically adjusts the knowledge graph structure according to the node weight, and optimizes the topological structure of the adaptive R & D knowledge graph through feedback iteration to generate a structured R & D document with a traceability relationship; The project data processing module is used to input the structured R & D documents into an R & D document understanding model trained based on a deep neural network. The R & D document understanding model extracts key R & D information according to the topological structure of the adaptive R & D knowledge graph, constructs a multi-level document framework based on the weight distribution of the key information, fills the information in the structured R & D documents into the multi-level document framework according to the information weights, performs document consistency verification, and generates project application documents.

[0011] In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the key information processing method for project R & D materials described above are implemented.

[0012] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the key information processing method for project R & D materials described above are implemented.

[0013] In a fifth aspect, the present application provides a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, the steps of the key information processing method for project R & D materials described above are implemented.

[0014] As can be seen from the above technical solutions, the present application provides a method and device for processing key information of project R & D materials. By constructing an R & D content recognition system and an adaptive R & D knowledge graph, through multi-modal information decomposition and fusion, unified processing and temporal alignment of text, voice, and image content are realized. Based on the R & D semantic analysis model, semantic completion and error correction are performed, the knowledge graph structure is dynamically constructed and optimized, and a structured document with traceability relationships is generated. The system uses a deep neural network model to extract key R & D information and constructs a multi-level document framework, realizing the intelligent conversion from R & D materials to project application documents. This method effectively solves the deficiencies of traditional technologies in multi-modal information processing and knowledge structure optimization, and significantly improves the efficiency of R & D materials management and project application. Description of the Drawings In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0015] Figure 1 It is a schematic flowchart of the method for processing key information of project R & D materials in the embodiments of the present application; Figure 2It is a structural diagram of a key information processing device for project R & D materials in an embodiment of the present application; Figure 3 It is a schematic structural diagram of an electronic device in an embodiment of the present application.

[0016] Reference numerals: Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. Detailed implementation manners To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without making creative efforts shall fall within the scope of protection of the present application.

[0017] In the technical solutions of the present application, the acquisition, storage, use, processing, etc. of data all comply with the relevant provisions of national laws and regulations.

[0018] Considering the problems existing in the prior art, the present application provides a method and device for processing key information of project R & D materials. By constructing a R & D content recognition system and an adaptive R & D knowledge graph. Through multi-modal information decomposition and fusion, unified processing and temporal alignment of text, voice, and image content are achieved. Based on the R & D semantic analysis model, semantic completion and error correction are performed, and the knowledge graph structure is dynamically constructed and optimized to generate a structured document with traceability relationships. The system uses a deep neural network model to extract key R & D information and constructs a multi-level document framework, realizing the intelligent conversion of R & D materials into project declaration documents. This method effectively solves the deficiencies of traditional technologies in multi-modal information processing and knowledge structure optimization, and significantly improves the efficiency of R & D material management and project declaration.

[0019] In order to effectively solve the deficiencies of traditional technologies in multi-modal information processing and knowledge structure optimization, and significantly improve the efficiency of R & D material management and project declaration, an embodiment of a method for processing key information of project R & D materials is provided in the present application. Refer to Figure 1 , the method for processing key information of project R & D materials specifically includes the following contents: Step S101: Import the enterprise R & D materials into the R & D content recognition system. Decompose the R & D materials through the R & D content recognition system to extract text content, speech content, and image content. Perform semantic segmentation on the text content, speaker separation and semantic recognition on the speech content, and scene understanding and text recognition on the image content. Align the text content, speech content, and image content in the time series dimension to construct a multi-modal information fusion matrix, and generate a semantic association vector based on the multi-modal information fusion matrix. Optionally, in this embodiment, a dedicated R & D content recognition system is first designed to process the multi-modal data generated during the enterprise R & D process. The system adopts a distributed architecture and includes three core modules: a file parser, a multi-modal feature extractor, and a feature fusion processor. The file parser supports the parsing of common document formats such as Word, PDF, and PPT, and can also process multimedia files such as audio records, technical drawings, and experimental pictures to ensure the complete import of R & D materials.

[0020] In the multi-modal information decomposition link of this embodiment, an innovative adaptive data stream separation mechanism is realized. First, determine the data type through file feature recognition, and then adopt corresponding parsing strategies according to different types. For mixed documents, the system uses a rule-based segmentation algorithm to accurately locate the boundaries of text, speech, and image content. During the processing, the association information between contents is retained through meta-data analysis, laying a foundation for subsequent time series alignment.

[0021] For the semantic segmentation of the text content in this embodiment, a bidirectional semantic analysis model based on BERT is designed. The model first performs tokenization on the input text to generate a sequence of word vectors. Then, context-related semantic features are extracted through multiple layers of Transformer encoders. The hidden layer dimension of the encoder is 768, and the number of attention heads is 12. In the semantic segmentation stage, a conditional random field (CRF) is used for sequence annotation, and the annotation granularity is dynamically adjusted according to the semantic integrity of the content.

[0022] In the speech processing link of this embodiment, an innovative speaker separation algorithm is realized. First, the fundamental frequency contour (F0) and Mel-frequency cepstral coefficients (MFCC) of the speech are obtained through acoustic feature extraction, and then the acoustic features are grouped using a clustering algorithm to identify the voiceprint features of different speakers. The speaker separation uses a deep clustering model, and the loss function of the model includes two parts: clustering loss and reconstruction loss: L = λ1·Lcluster + λ2·Lrecon, where λ1 and λ2 are weight coefficients.

[0023] In this embodiment, a multi-task learning framework is designed for image content analysis. This framework simultaneously executes scene understanding and text recognition tasks, sharing the underlying feature extraction network. For scene understanding, an improved ResNet50 model is adopted, enhancing the ability to capture multi-scale features through spatial pyramid pooling. The text recognition task uses an attention-based sequence recognition model, which can accurately identify information such as technical parameters and labeled text in the image.

[0024] In this embodiment, a multi-modal alignment mechanism based on time series is innovatively implemented. First, the timestamp information of each modal content is extracted to construct a time series feature sequence. Then, the optimal alignment path between different modal sequences is calculated through the dynamic time warping (DTW) algorithm. The objective function of the alignment process is: DTW(X,Y) = min(Σd(xi,yi)), where d(xi,yi) represents the distance metric between the two sequences at the i-th time point.

[0025] In this embodiment, an efficient method for constructing a multi-modal information fusion matrix is designed. First, the features of different modalities are mapped to a unified feature space, and the feature dimension is uniformly set to 512. Then, the interaction relationship between modalities is calculated through the multi-head attention mechanism. The calculation formula for the attention weight is: α = softmax(QK^T / √d), where Q and K are the query matrix and the key matrix respectively, and d is the feature dimension. Finally, the fused feature matrix is generated through weighted combination.

[0026] In the semantic association vector generation stage of this embodiment, the main semantic features are extracted by means of tensor decomposition. Specifically, the Tucker decomposition method is used to decompose the high-dimensional fusion matrix into the product of a core tensor and factor matrices. By setting appropriate rank parameters, the most important semantic information is retained while significantly reducing the dimension of the features. The generated semantic association vector not only retains the semantic content of multi-modal information but also has good computational efficiency.

[0027] Through the above technological innovations, this embodiment effectively solves several key problems in the processing of enterprise R & D materials: inaccurate parsing and separation of multi-modal data, difficult speaker recognition of speech content, insufficient semantic understanding of graphic and text information, inaccurate alignment of multi-modal information, etc. In practical applications, this solution can accurately process various R & D documents, realize the intelligent parsing and fusion of multi-modal information, and provide high-quality semantic association vectors for the subsequent construction of knowledge graphs. It is particularly suitable for processing enterprise R & D materials with complex structures and diverse forms, significantly improving the accuracy and processing efficiency of information extraction.

[0028] Step S102: Construct an adaptive R & D knowledge graph, input the semantic association vector into the R & D semantic analysis model. The R & D semantic analysis model performs semantic completion and error correction on the semantic association vector based on the context relationship, generates normalized semantic units, constructs the normalized semantic units into knowledge graph nodes, calculates the node weights based on the association degree between the knowledge graph nodes, dynamically adjusts the knowledge graph structure according to the node weights, and optimizes the topological structure of the adaptive R & D knowledge graph through feedback iteration to generate a structured R & D document with traceability relationships. Optionally, in this embodiment, a dedicated R & D semantic analysis model is first constructed, and an improved Transformer architecture is used to process the semantic association vector. The model includes multiple layers of bidirectional encoders, with 8 attention heads set in each layer and a hidden layer dimension of 512. A special position encoding is designed in the input layer, which can retain the timing information in the R & D process: PE(pos,2i) =sin(pos / 10000^(2i / d)), PE(pos,2i + 1) = cos(pos / 10000^(2i / d)), where pos represents the sequence position, i represents the dimension position, and d is the model dimension.

[0029] In this embodiment, an innovative semantic completion mechanism is realized according to the characteristics of the R & D field. First, a domain ontology library is constructed, which includes key concepts, attribute relationships, and constraint rules in the R & D process. A context-aware completion algorithm is designed based on the ontology library, and the semantic relevance between the position to be completed and the known content is calculated through the attention mechanism: attention(q,k,v) =softmax(qk^T / √d)v, where q is the query vector, k is the key vector, v is the value vector, and d is the vector dimension. The continuity and logical consistency of the technical route are considered in the completion process.

[0030] In this embodiment, an efficient semantic error correction module is designed. The module adopts a multi-task learning framework and simultaneously performs tasks such as spelling check, grammar correction, and semantic calibration. For the error correction of professional terms, a fuzzy matching algorithm based on the edit distance is introduced, and a domain dictionary is combined for verification. The semantic calibration task is realized through a pre-trained language model, and the model is trained for domain adaptability on a large-scale R & D document.

[0031] In this embodiment, an innovative generation mechanism for normalized semantic units is realized. First, the semantic content is hierarchically decomposed to extract core concepts, key attributes, and relationship descriptions. Then, through a template-guided generation network, the decomposed content is recombined into a standardized semantic expression. The loss function of the generation network includes a content preservation loss and a structure normalization loss: L =λ1·Lcontent + λ2·Lstructure, where λ1 and λ2 are balance coefficients.

[0032] In this embodiment, a dynamic node mapping algorithm is designed in the knowledge graph construction phase. When converting the normalized semantic units into graph nodes, entity types, attribute features, and relationship constraints are considered. The feature vector of a node is generated by a multi-layer perceptron: h = MLP([e;a;r]), where e is the entity embedding, a is the attribute embedding, and r is the relationship embedding. This representation preserves the complete information of the semantic units.

[0033] This embodiment implements an innovative method for calculating node correlation. First, the structural features between nodes are extracted based on a graph attention network, and then the comprehensive correlation strength is calculated by combining semantic similarity. The formula for the correlation is: score =α·structural_sim + (1-α)·semantic_sim, where α is a dynamically adjusted weight coefficient, structural_sim is the structural similarity, and semantic_sim is the semantic similarity.

[0034] For the dynamic optimization of the knowledge graph, this embodiment designs an adaptive structure adjustment algorithm. Based on a reinforcement learning framework, the graph optimization is modeled as a sequential decision-making problem. The state space includes the topological features of the current graph, and the action space includes operations such as node merging, splitting, and connection. The reward function is designed as: R = w1·coverage + w2·consistency + w3·compactness, where coverage represents the knowledge coverage, consistency represents the structural consistency, and compactness represents the graph compactness.

[0035] This embodiment designs a feedback iterative optimization mechanism. In each iteration, the system evaluates the quality metrics of the current graph structure, including connectivity, coverage, and interpretability. Based on the evaluation results, the optimization strategy is dynamically adjusted to update the node weights and the connection relationships of the edges. The termination condition of the iterative process is determined based on the convergence state of the graph quality.

[0036] In the structured document generation phase, this embodiment implements a graph-based content organization method. First, the document framework is determined according to the hierarchical structure of the knowledge graph, and then the content order is arranged based on the node weights and relationship strengths. For content with a traceability relationship, the system ensures the integrity and traceability of the information by backtracking the association paths in the graph.

[0037] Through the above technical innovations, this embodiment effectively solves several key problems in the construction of the R & D knowledge graph: non-standard semantic expressions, incomplete knowledge associations, unreasonable graph structures, difficulties in content traceability, etc. In practical applications, this solution can accurately understand and organize the knowledge content in R & D documents, establish a clear knowledge association network, and provide a reliable knowledge basis for the generation of project application documents. It is particularly suitable for the knowledge management of complex R & D projects, significantly improving the accuracy of knowledge extraction and organization, and enhancing the traceability of document content. The adaptive characteristics of this solution enable it to handle R & D knowledge in different fields and have broad application prospects.

[0038] Step S103: Input the structured R & D document into an R & D document understanding model trained based on a deep neural network. The R & D document understanding model extracts key R & D information according to the topological structure of the adaptive R & D knowledge graph, constructs a multi-level document framework based on the weight distribution of the key information, fills the information in the structured R & D document into the multi-level document framework according to the information weights, performs document consistency verification, and generates a project application document.

[0039] Optionally, in this embodiment, a special R & D document understanding model is first designed, and a multi-layer neural network architecture is used to process structured R & D documents. The basic architecture of this model includes a word embedding layer, a feature extraction layer, and a semantic understanding layer. The word embedding layer uses pre-trained domain-adaptive word vectors to map professional terms and technical descriptions in the document into dense vector representations. The feature extraction layer uses a bidirectional long short-term memory network (BiLSTM) structure with a hidden layer dimension set to 512, which can effectively capture long-distance semantic dependencies.

[0040] In the knowledge graph feature processing link of this embodiment, a graph structure feature extraction mechanism is innovatively implemented. First, the topological structure of the adaptive R & D knowledge graph is converted into an adjacency matrix representation, and then structured features between nodes are extracted through a graph attention network (GAT). The calculation of the attention coefficient adopts a shared parameter method: αij = softmax(LeakyReLU(a^T[Whi||Whj])), where W is the weight matrix, hi and hj are node feature vectors, a is the attention vector, and || represents the concatenation operation.

[0041] This embodiment designs a deep feature fusion module to perform multi-level fusion of document features and graph features. The fusion process uses a gating mechanism to dynamically adjust the importance of different features: g = σ(Wg[fd;fg]), where fd is the document feature, fg is the graph feature, Wg is the learnable weight matrix, and σ is the sigmoid activation function. The fused features retain both the semantic information of the document and the structured information of the knowledge graph.

[0042] In this embodiment, for the extraction of key R & D information, an information recognition framework based on multi-task learning is implemented. This framework simultaneously performs entity recognition, relation extraction, and attribute prediction tasks, sharing underlying feature representations. The loss functions between tasks are designed as: L = λ1·Lentity + λ2·Lrelation + λ3·Lattribute, where λ1, λ2, and λ3 are task weight coefficients, which are dynamically adjusted through the validation set.

[0043] In this embodiment, a multi-level document framework generation algorithm is innovatively constructed. First, hierarchical clustering is performed based on the weight distribution of key information, using an improved DBSCAN algorithm, and the density threshold is dynamically set according to the information weight. The clustering results form a tree-like hierarchical structure, and each level corresponds to a different level of the document framework. The relationship between levels is evaluated through information entropy and mutual information: I(X;Y) = H(X) - H(X|Y), where H(X) is the information entropy and H(X|Y) is the conditional entropy.

[0044] In this embodiment, an intelligent information filling strategy is designed. For each document framework position, the system calculates the matching degree between the candidate information and the position requirements: score = w1·relevance + w2·importance + w3·coherence, where relevance represents content relevance, importance represents information importance, coherence represents context coherence, and w1, w2, and w3 are weight coefficients. The information with the highest score is selected for filling.

[0045] In this embodiment, an innovative document consistency verification mechanism is implemented. First, information redundancy is detected by calculating semantic similarity: sim(x,y) = cos(Wx,Wy), where W is the semantic projection matrix. The content with a similarity exceeding the threshold is merged and optimized. Then, a pre-trained language model is used to evaluate the coherence of the document, including two levels: local coherence and global coherence.

[0046] In this embodiment, a document optimization algorithm is designed according to the characteristics of project declarations. This algorithm includes three core steps: structure optimization, content optimization, and format optimization. Structure optimization ensures that the document has a clear hierarchy and reasonable logic; content optimization guarantees accurate expression and prominent key points; format optimization ensures compliance with declaration requirements, being beautiful and standard. The optimization process adopts an iterative method, and the document quality is evaluated after each round of iteration until the preset standard is reached.

[0047] In this embodiment, an adaptive content organization mechanism is implemented in the document generation process. According to the project type and application requirements, the organization method and expression form of the content are dynamically adjusted. For technology innovation projects, the focus is on highlighting technological advancement and innovation; for industrialization projects, market value and implementation feasibility are emphasized; for basic research projects, theoretical innovation and research methods are highlighted.

[0048] Through the above technological innovations, this embodiment effectively solves multiple key problems in the generation of project application documents: unreasonable document structure, unclear key points, redundant content, non-standard expression, etc. In practical applications, this solution can accurately understand the content of R & D documents, extract key information, and automatically generate project application documents that meet the requirements. It is particularly suitable for the application work of enterprise R & D projects, significantly improving the quality and efficiency of document generation, and reducing the workload of manual writing and review. The adaptive nature of this solution enables it to handle different types of project application requirements, with broad application value.

[0049] As can be seen from the above description, the method for processing key information of project R & D materials provided by the embodiment of this application can build a R & D content recognition system and an adaptive R & D knowledge graph. Through multi-modal information decomposition and fusion, unified processing and temporal alignment of text, speech, and image content are achieved. Based on the R & D semantic analysis model, semantic completion and error correction are performed, the knowledge graph structure is dynamically constructed and optimized, and a structured document with traceability relationships is generated. The system uses a deep neural network model to extract key R & D information and constructs a multi-level document framework, realizing the intelligent transformation from R & D materials to project application documents. This method effectively solves the deficiencies of traditional technologies in multi-modal information processing and knowledge structure optimization, significantly improving the efficiency of R & D material management and project application.

[0050] In an embodiment of the method for processing key information of project R & D materials of this application, the following specific content may also be included: Step S201: Classify and store R & D materials in digital format into a preset file directory according to the file format type, read the file content in the preset file directory based on a file parser, parse the file content into an initial data stream according to the data structure, divide the initial data stream into a text data stream, an audio data stream, and an image data stream through the R & D content recognition system, perform text encoding conversion and format standardization processing on the text data stream, perform audio noise reduction and waveform normalization processing on the audio data stream, and perform image denoising and size standardization processing on the image data stream; Step S202: Based on the semantic segmentation model, the text data stream is segmented into independent semantic segments. The voiceprint features of the audio data stream are extracted to construct a speaker voiceprint library. The audio data stream is transcribed into text content using a speech recognition model. Based on a convolutional neural network model, scene classification and object detection are performed on the image data stream. The text contained in the image is converted into text format using an optical character recognition model. The semantic segments, transcribed text, and recognized text are combined to construct a unified text corpus.

[0051] Optionally, in this embodiment, a dedicated file format classification and storage mechanism is first designed. In view of the diversity of R & D materials, a hierarchical file directory structure is constructed, including technical document directories, experimental record directories, meeting minutes directories, etc. File classification adopts a multi-level classification strategy based on file features, and the features include file extensions, metadata information, and content features. For compound format files, the system extracts internal structure features through in-depth scanning to ensure accurate classification.

[0052] This embodiment implements an intelligent file parser. The parser adopts a plug-in architecture and supports dynamic loading of parsing modules for common formats such as Word, PDF, and PPT. The hierarchical structure, format information, and metadata of the document are retained during the parsing process, and the parsing results are stored in a unified data structure: Content = {metadata, structure, content_blocks}, where metadata contains file attributes, structure describes the document structure, and content_blocks stores the actual content.

[0053] In the data stream division section of this embodiment, a multi-modal content recognition mechanism is innovatively implemented. First, the content features are analyzed through a feature extraction network, and then the modality is judged based on the multi-classifier integration. The classifier integration includes CNN, RNN, and a rule engine, and the final decision is determined by weighted voting: vote = Σwi·ci, where wi is the classifier weight and ci is the classification result. For mixed content, the system can accurately locate the boundaries of different modalities.

[0054] For the processing of the text data stream in this embodiment, an adaptive encoding conversion mechanism is designed. First, the text encoding method is identified through an encoding detector, and then intelligent transcoding is performed to ensure the correct conversion of special characters and professional symbols. Format standardization adopts a rule-based processing pipeline, including steps such as blank character normalization, punctuation unification, and paragraph format adjustment.

[0055] In this embodiment, an innovative noise reduction algorithm is implemented in the audio processing section. This algorithm combines frequency domain analysis and deep learning methods. First, the time-frequency spectrogram is obtained through the short-time Fourier transform. Then, the U-Net structure is used to predict the noise mask. Finally, noise reduction is achieved through Wiener filtering. Waveform normalization adopts dynamic range compression to maintain the consistency of the audio signal: y = sign(x)|x|^α, where x is the input signal and α is the compression coefficient.

[0056] In this embodiment, an efficient image preprocessing process is designed. Image denoising uses an improved non-local mean algorithm to improve the processing efficiency through an adaptive search window. Size normalization takes into account the image content features, and different scaling strategies are adopted for the text area and the chart area to ensure the clarity of key information: scale = min(target_size / original_size, max_scale).

[0057] In this embodiment, in the semantic segmentation section, a sequence annotation model based on BERT is adopted. The model input includes the text sequence and the position encoding, and the context features are extracted through multiple layers of Transformer encoders. The prediction of the segmentation points is optimized by conditional random fields, considering semantic integrity and syntactic structure: P(Y|X) = exp(Σθk·fk(X,Y)) / Z, where θk is the feature weight and fk is the feature function.

[0058] In this embodiment, an innovative method for extracting voiceprint features is implemented. First, the fundamental frequency features and acoustic features of the audio are extracted, and then the voiceprint vector is generated through a deep neural network. The voiceprint library adopts a hierarchical storage structure, supporting fast retrieval and dynamic update. For new speakers, the system updates the voiceprint model through incremental learning.

[0059] In this embodiment, in the speech recognition section, a recognition model adapted to the R & D scenario is designed. The model adopts the Transformer-CTC architecture and is optimized for the recognition of professional vocabulary and terms. The decoding process combines domain knowledge constraints to improve the recognition accuracy of professional terms: score = λ1·acoustic_score + λ2·language_score + λ3·domain_score.

[0060] In this embodiment, for image content analysis, a multi-task learning framework is implemented. The framework includes three branches: scene classification, object detection, and text recognition, sharing a basic feature extraction network. Text recognition uses an attention-based sequence recognition model, and the recognition effect of structured text such as tables and formulas is particularly optimized.

[0061] This embodiment designs an intelligent text corpus construction mechanism. First, the text from different sources is unified in format and duplicate removed, and then organized based on topic similarity. The corpus supports incremental updates and version management to ensure data consistency and traceability.

[0062] Through the above technological innovations, this embodiment effectively solves several key problems in the processing of R & D materials: difficult parsing caused by complex file formats, inaccurate recognition of multi-modal content, unstable audio and video quality, incomplete text extraction, etc. In practical applications, this solution can efficiently process various R & D documents, achieve standardized storage and processing of data, and provide high-quality input for subsequent semantic analysis. It is particularly suitable for batch processing of large-scale R & D materials, significantly improving the accuracy and efficiency of data processing. The adaptive characteristics of this solution enable it to process R & D materials from different sources and formats, with broad application value.

[0063] In an embodiment of the method for processing key information of project R & D materials in this application, the following content may also be specifically included: Step S301: Perform chronological sorting on the text content, speech content, and image content based on timestamps, calculate the time interval between adjacent contents, use a bidirectional long short-term memory network to extract the chronological features of each modal content, construct a chronological feature vector, perform normalization processing on the chronological feature vector, calculate the chronological alignment degree between different modal contents based on the dynamic time warping algorithm, and mark the content with a chronological alignment degree higher than the preset threshold as an associated segment; Step S302: Map the associated segment to the feature space to construct a feature matrix, use the attention mechanism to calculate the association weights between different modal features, perform weighted fusion on the feature matrix based on the association weights to obtain a multi-modal information fusion matrix, and use the tensor decomposition method to perform dimensionality reduction processing on the multi-modal information fusion matrix, and extract the main feature components to construct a semantic association vector.

[0064] Optionally, this embodiment first designs a dedicated chronological data processing mechanism. When processing R & D documents, a unified timeline is established by extracting file metadata and content timestamps. For content lacking timestamps, it is inferred and supplemented through context information and file creation time. The time interval between adjacent contents is calculated by Δt = ti+1 - ti, where ti represents the timestamp of the i-th content. This calculation method can reflect the chronological relevance of content in the R & D process.

[0065] This embodiment implements a temporal feature extraction network based on BiLSTM. The network consists of three parallel branches that process text, speech, and image data respectively. Each BiLSTM layer in each branch contains 256 hidden units, which capture long-distance dependencies through forward and backward propagation. The network input is processed by a feature embedding layer, which maps data of different modalities into a feature space of a unified dimension to ensure the consistency of temporal features.

[0066] This embodiment innovatively designs a normalization method for temporal feature vectors. First, the mean and standard deviation of the feature vectors are calculated, and then the feature distribution is adjusted through distribution normalization: x' = (x - μ) / (σ + ε), where μ is the mean, σ is the standard deviation, and ε is a smoothing factor. This normalization process can eliminate the scale differences of data in different modalities and improve the accuracy of subsequent alignment.

[0067] This embodiment implements an improved dynamic time warping algorithm for the temporal alignment problem. The algorithm calculates the optimal alignment path by constructing an accumulated distance matrix and introduces local constraint conditions to limit the path search range. The calculation of the alignment degree takes into account the time distance and content similarity: align_score = w1·time_sim + w2·content_sim, where w1 and w2 are weight coefficients that are dynamically adjusted according to data characteristics.

[0068] This embodiment designs an intelligent associated segment marking mechanism. An adaptive threshold is set according to the alignment degree, and the threshold value is determined through statistical analysis: threshold = μ + k·σ, where μ and σ are the mean and standard deviation of the alignment degree respectively, and k is an adjustment coefficient. For special scenarios, such as experimental records and meeting minutes, the system will appropriately lower the threshold requirement to ensure that key information is not missed.

[0069] This embodiment implements an innovative feature transformation method in the feature space mapping link. A multi-layer perceptron is used for non-linear mapping, and the network structure includes three hidden layers with dimensions of 512, 256, and 128 in sequence. The feature transmission is improved through residual connection and layer normalization to ensure the training stability of the deep network. The mapped features retain the semantic information of the original data and have better comparability at the same time.

[0070] This embodiment designs a multi-head attention mechanism for the correlation analysis of different modality features. The number of attention heads is set to 8, and each attention head independently calculates the correlation relationship between features. The attention weight is calculated through scaled dot-product attention: attention = softmax(QK^T / √d), where Q is the query matrix, K is the key matrix, and d is the feature dimension. This mechanism can capture the complex interaction relationships between different modality features.

[0071] This embodiment realizes an efficient feature fusion strategy. First, the feature matrix is weighted and combined based on attention weights, and then the information flow is controlled through a gating mechanism: gate = σ(W[x;y]), where x and y are the features to be fused, and W is a learnable weight matrix. This method can adaptively adjust the importance of different modal information and generate more representative fused features.

[0072] This embodiment innovatively uses tensor decomposition for feature dimensionality reduction. Specifically, the Tucker decomposition method is used to decompose the high-dimensional fusion matrix into the product of a core tensor and factor matrices. The rank parameter of the decomposition is determined through cross-validation, which significantly reduces the feature dimension while retaining the main information. The reconstructed error analysis ensures that the features after dimensionality reduction can still accurately express the original semantics.

[0073] This embodiment designs a method for constructing a semantic association vector. The dimensionality-reduced features are projected into the semantic space through a non-linear projection, and the relative distance relationship between the features is maintained during the projection process. The dimension of the association vector is set to 64, which can improve the computational efficiency while maintaining the expressive ability.

[0074] Through the above technological innovations, this embodiment effectively solves several key problems in the processing of R & D documents: inaccurate temporal alignment of multi-modal data, insufficient feature fusion, dimensionality disaster, etc. In practical applications, this solution can accurately identify and associate R & D information of different modalities, achieve high-quality feature extraction and fusion, and provide a reliable semantic basis for the subsequent construction of the knowledge graph. It is particularly suitable for the processing of documents in complex R & D projects, significantly improving the accuracy and efficiency of multi-modal information processing. The adaptive characteristics of this solution enable it to process different types of R & D documents and have broad application prospects.

[0075] In an embodiment of the method for processing key information of project R & D materials in this application, the following content may also be specifically included: Step S401: Construct an ontology library for the R & D field, use the concept nodes, relationship types, and attribute constraints in the ontology library as the basic framework of the knowledge graph, map the semantic association vector to the graph space based on the knowledge graph construction rules, use a named entity recognition model to identify entity types from the semantic association vector, extract the semantic relationships between entities using a relationship extraction model, and integrate the identified entities and relationships into an initial knowledge graph; Step S402: Use a bidirectional Transformer model to perform context encoding on the semantic association vector, calculate the semantic correlation degree between tokens based on attention scores, complete the missing semantic information in the semantic association vector, construct a semantic error correction model to correct ambiguous semantic expressions, and construct a normalized semantic unit with the completed and corrected semantic information according to preset rules.

[0076] Optionally, in this embodiment, a dedicated mechanism for constructing an ontology library in the R & D field is first designed. In view of the characteristics of R & D projects, the ontology library adopts a multi-level structure, including a core concept layer, a relationship type layer, and an attribute constraint layer. The core concept layer covers basic concepts such as R & D goals, technical solutions, innovation points, and experimental methods; the relationship type layer defines relationships such as dependence, inheritance, and composition between concepts; the attribute constraint layer specifies the necessary attributes and value ranges of each concept. The ontology library is constructed by combining expert knowledge and machine learning to ensure the integrity and accuracy of domain knowledge.

[0077] This embodiment implements an innovative semantic association vector mapping method. First, a mapping rule library is constructed, which contains the corresponding relationships from vector features to graph concepts. The mapping process uses an attention mechanism to calculate the similarity between the vector and the concept node: sim(v, c) = v^T·Wc, where v is the vector representation, c is the concept representation, and W is a learnable mapping matrix. By setting a similarity threshold, the accuracy of the mapping is ensured.

[0078] For the entity recognition task, this embodiment designs a named entity recognition model based on BERT. The model is fine-tuned on the R & D document dataset to support the recognition of professional entities including technical terms, parameter indicators, experimental equipment, etc. The BIO annotation scheme is adopted, and the annotation sequence is optimized through the CRF layer: P(Y|X) = exp(Σθi·fi(X,Y)) / Z, where θi is the feature weight, fi is the feature function, and Z is the normalization factor.

[0079] This embodiment innovatively implements a relation extraction model. The model adopts a distant supervision learning framework and automatically constructs training data using an existing knowledge base. The model architecture includes an entity encoder and a relation classifier, and processes noise samples through a selective attention mechanism. For complex multi-hop relations, the system supplements them through path reasoning to improve the integrity of relation extraction.

[0080] This embodiment designs an intelligent knowledge graph integration strategy. First, the identified entities are disambiguated and merged, and aligned with the existing knowledge base through entity linking technology. Then, entity relations are filtered based on a credibility score, and the scoring function comprehensively considers the frequency of the relation, context support, and expert rules. The integration process adopts an incremental update mechanism to ensure the dynamic expansion ability of the knowledge graph.

[0081] In the context encoding link of the semantic association vector in this embodiment, an improved bidirectional Transformer architecture is adopted. The model contains 6 encoder layers, with 12 attention heads set in each layer, and the hidden layer dimension is 768. The position encoding adopts a relative position representation method to better capture long-distance dependency relationships. The model is pre-trained on a large-scale R & D document and has strong semantic understanding ability.

[0082] This embodiment realizes an efficient semantic relevance calculation mechanism. The correlation strength between tokens is calculated through the multi-head self-attention mechanism, and each attention head focuses on different semantic features. The relevance calculation adopts the scaled dot-product method and introduces a position-aware bias term to improve the calculation accuracy. For professional terms and technical descriptions, the system assigns higher attention weights.

[0083] For the semantic completion task, this embodiment designs a context-based generation model. The model adopts an encoder-decoder architecture and learns semantic completion through masked prediction. The completion process takes into account domain knowledge constraints to ensure the professionalism and rationality of the generated content. For uncertain completion results, the system generates multiple candidates for selection.

[0084] This embodiment innovatively constructs a semantic error correction model. The model combines a rule base and deep learning methods. First, common errors are identified through rule matching, and then a pre-trained language model is used for deep semantic correction. The error correction process takes into account the particularity of professional terms to avoid incorrect modification of correct professional expressions.

[0085] This embodiment designs a method for constructing standardized semantic units. Based on a preset semantic template, the completed and corrected information is organized into a standardized expression form. The template design takes into account the characteristics of R & D documents, including various types such as technical descriptions, parameter explanations, and experimental processes. The construction process ensures the standardization of semantic units through quality inspection.

[0086] Through the above technological innovations, this embodiment effectively solves several key problems in R & D document processing: non-standard domain knowledge expression, incomplete entity relationship extraction, inaccurate semantic understanding, difficult processing of professional terms, etc. In practical applications, this solution can accurately identify and organize professional knowledge in R & D documents, establish a complete knowledge association network, and provide a reliable knowledge basis for subsequent document generation. It is particularly suitable for knowledge management in technology-intensive projects and significantly improves the accuracy of knowledge extraction and organization. The adaptive characteristics of this solution enable it to process R & D documents in different fields and have broad application value.

[0087] In an embodiment of the method for processing key information in the project R & D materials of this application, the following content may also be specifically included: Step S501: Map the standardized semantic units into knowledge graph nodes according to the node construction rules, extract feature vectors for the knowledge graph nodes, calculate the semantic similarity between nodes using a graph attention network, construct a node adjacency matrix, calculate the node importance based on node degree centrality and eigenvector centrality, combine the node importance with the semantic similarity to obtain a node weight value, and use the node weight value as the edge weight of the knowledge graph; Step S502: Construct a graph optimization objective function based on the edge weights, and use the gradient descent method to iteratively optimize the topological structure of the knowledge graph. In each iteration, evaluate the connectivity and coverage of the knowledge graph. When the optimization objective function converges, obtain a stable knowledge graph structure, hierarchically organize the node information based on the knowledge graph structure, and convert the organized node information into a structured R & D document according to a preset template.

[0088] Optionally, in this embodiment, a special node mapping mechanism is first designed. According to the characteristics of the R & D document, detailed node construction rules are formulated, including node type definition, attribute specification, and relationship constraints. The mapping process uses semantic similarity matching, and calculates the matching degree between the normalized semantic unit and the existing node through cosine similarity: sim(u, v) = u^T·v / (||u||·||v||), where u and v are the vector representations of the semantic unit and the node respectively. For the new semantic unit, the system will automatically create a new node and inherit the corresponding attribute structure.

[0089] This embodiment realizes an innovative node feature extraction method. Adopt a graph representation learning framework, and generate feature vectors by combining the attribute information and structure information of the nodes. The feature extraction process includes two parts: an attribute encoder and a structure encoder, and dynamically fuses features from different sources through an attention mechanism. For the professional concepts in the R & D document, the system will additionally introduce domain knowledge to enhance the feature representation.

[0090] This embodiment designs a calculation model based on a graph attention network for calculating the semantic similarity between nodes. The model contains multiple attention layers, and each layer contains 8 attention heads, which can capture the semantic associations between nodes from different angles. The calculation of the attention score takes into account the local structure and global context of the nodes: α = LeakyReLU(W[hi||hj]), where hi and hj are the node features, and W is a learnable weight matrix.

[0091] This embodiment innovatively realizes a node importance calculation mechanism. First, calculate the degree centrality of the node, which reflects the direct connection importance of the node in the graph. Then capture the influence of the node in the global network through eigenvector centrality. The two centrality indicators are combined through weighted combination to obtain the comprehensive importance: importance = w1·degree_cent + w2·eigenvector_cent, where w1 and w2 are dynamically adjusted weight coefficients.

[0092] In this embodiment, an intelligent edge weight assignment strategy is designed. By combining node importance and semantic similarity, an edge weight calculation formula is constructed: weight = α·importance + (1-α)·similarity, where α is a balance factor. This method not only considers the structural importance of nodes but also retains the strength of semantic associations, enabling the graph structure to better reflect the organizational characteristics of knowledge.

[0093] In the graph optimization phase of this embodiment, an innovative objective function design is achieved. The objective function consists of three components: structural integrity, semantic consistency, and coverage: L = λ1·Lstructure + λ2·Lsemantic + λ3·Lcoverage, where λ1, λ2, and λ3 are weight coefficients. By adjusting these coefficients, the importance of different optimization objectives can be balanced.

[0094] This embodiment uses an improved gradient descent algorithm for graph optimization. The algorithm employs an adaptive learning rate strategy, dynamically adjusting the learning step size according to the changes in the objective function. To prevent getting stuck in local optima, a simulated annealing mechanism is introduced, allowing the algorithm to perform a certain degree of random exploration during the optimization process.

[0095] In this embodiment, a comprehensive graph evaluation mechanism is designed. In each iteration, the system evaluates the graph quality from two dimensions: connectivity and coverage. Connectivity evaluation uses clustering coefficient and shortest path analysis, and coverage evaluation is based on comparison with the domain ontology. When the evaluation metrics change less than a preset threshold for multiple consecutive rounds, the optimization process is considered to have converged.

[0096] This embodiment implements an efficient hierarchical organization method. Based on the optimized graph structure, a community discovery algorithm is used to identify knowledge modules, and then a hierarchical structure of knowledge is constructed through hierarchical clustering. The clustering process takes into account the edge weights and semantic associations between nodes to ensure the rationality of the division.

[0097] In this embodiment, a document conversion mechanism is innovatively designed. According to the characteristics of R & D documents, multiple document templates are preset, including technical solutions, experimental reports, evaluation reports, etc. In the conversion process, the document type is first determined, and then the content is organized based on the hierarchical structure. A standardized document structure is generated through template guidance.

[0098] Through the above technological innovations, this embodiment effectively solves several key problems in the construction of the R & D knowledge graph: inaccurate node mapping, low efficiency of structure optimization, unreasonable knowledge organization, non-standard document generation, etc. In practical applications, this solution can accurately construct and optimize the graph representation of R & D knowledge, realize the effective organization and documented expression of knowledge, and provide strong support for project management and knowledge reuse. It is particularly suitable for the knowledge management of complex R & D projects, significantly improving the accuracy of knowledge organization and the quality of document generation. The adaptive characteristics of this solution enable it to handle different types of R & D projects, with broad application prospects.

[0099] In an embodiment of the method for processing key information of project R & D materials in this application, the following content may also be specifically included: Step S601: Construct a multi-layer perceptron neural network as the basic architecture of the document understanding model, convert the structured R & D document into a sequence of word vectors and input it into the neural network. Set a convolutional layer in the neural network to extract local features, set a pooling layer to reduce the feature dimension, set a fully connected layer to fuse the feature information, use the batch normalization method to standardize the feature data, and introduce a non-linear transformation through an activation function to obtain a deep feature representation of the document. Step S602: Convert the topological structure of the adaptive R & D knowledge graph into a graph structure feature vector, use a graph convolutional network to extract features from the graph structure feature vector, fuse the extracted features with the deep document features, calculate the relevance between the fused features and the R & D key information based on the attention mechanism, and select the features with a relevance higher than the preset threshold as the R & D key information.

[0100] Optionally, in this embodiment, a dedicated multi-layer perceptron neural network architecture is first designed. Considering the characteristics of R & D documents, the network adopts a multi-branch structure. The main network contains 5 hidden layers with dimensions of 1024, 512, 256, 128, and 64 in sequence. The input layer designs a special word vector mapping mechanism and uses a pre-trained domain word vector model, which is trained on a large-scale R & D document corpus and can accurately express the semantic features of professional terms. The word vector dimension is set to 300, and the word order information is retained through position encoding.

[0101] This embodiment realizes an innovative convolutional feature extraction mechanism. Multi-scale convolutional kernels are set, including window sizes of 2, 3, 4, and 5 words, and the number of convolutional kernels of each scale is 128. The convolutional operation uses causal convolution to ensure that the feature extraction process considers context dependencies: F(s) = Σ(k·x[s - i]), where k is the convolutional kernel parameter, x is the input sequence, and s is the position index. This design can effectively capture text features of different granularities.

[0102] In this embodiment, an adaptive pooling strategy is designed for feature dimensionality reduction. A max pooling layer is set after each convolutional branch, and the pooling window size is dynamically adjusted according to the size of the feature map. At the same time, an attention pooling mechanism is introduced to calculate the importance weights of features and retain the most representative features. The pooling operation significantly reduces the feature dimension while maintaining the integrity of key information.

[0103] This embodiment innovatively implements a multi-layer feature fusion mechanism. Through residual connections and gating mechanisms, effective combination of features at different levels is achieved. The design of the gating unit takes into account the importance and complementarity of features: g = σ(Wg·[h1;h2]), where h1 and h2 are the features to be fused, and Wg is a learnable weight matrix. In this way, the contribution degrees of different features can be adaptively adjusted.

[0104] In the feature normalization step of this embodiment, an improved batch normalization method is adopted. Considering the distribution characteristics of professional terms and numerical indicators in the R & D documents, adaptive normalization parameters are designed. The normalization process not only considers the statistical characteristics within the batch but also introduces global statistical information, improving the stability of processing.

[0105] This embodiment designs a non-linear feature transformation strategy. In each layer of the network, the GELU activation function is used, which has a smoother gradient characteristic compared with the traditional ReLU. For deep features, the gradient propagation is improved through the residual structure to avoid the problem of network degradation. This design significantly enhances the expression ability of the model.

[0106] For the feature extraction of the knowledge graph in this embodiment, an innovative graph structure encoding method is implemented. First, the topological structure of the graph is converted into an adjacency matrix and a node feature matrix, and then information is transmitted through a graph convolutional network. The design of the graph convolution operation takes into account the degree distribution of nodes and the weights of edges: H' = σ(D^(-1 / 2)AD^(-1 / 2)HW), where A is the adjacency matrix, D is the degree matrix, H is the node feature, and W is a learnable parameter.

[0107] This embodiment innovatively designs a feature fusion strategy. The multi-head attention mechanism is used to calculate the interaction relationship between document features and graph features. The number of attention heads is set to 8, and each head independently learns different feature association patterns. The fusion process is realized through a gated memory network, which can maintain important feature information in the long term.

[0108] This embodiment implements an efficient key information recognition mechanism. The relevance to a predefined key information template is calculated based on the fused features. The relevance calculation adopts a weighted combination of cosine similarity and attention scores. Feature screening is performed by setting a dynamic threshold, and the threshold value is adaptively adjusted according to the feature distribution.

[0109] This embodiment designs an intelligent feature selection method. During the screening process, not only the relevance of features is considered, but also a diversity constraint is introduced to ensure that the selected features can comprehensively cover the key content of R & D documents. The selection algorithm adopts a greedy strategy to gradually select the optimal features while maintaining the complementarity between features.

[0110] Through the above technological innovations, this embodiment effectively solves several key problems in R & D document understanding: insufficient extraction of document features, insufficient utilization of knowledge graph information, inaccurate feature fusion, incomplete identification of key information, etc. In practical applications, this solution can accurately understand the content of R & D documents, extract valuable key information, and provide a reliable information basis for the generation of project application documents. It is particularly suitable for document processing of technology-intensive projects, significantly improving the accuracy and integrity of information extraction. The adaptive characteristics of this solution enable it to process different types of R & D documents, with broad application prospects.

[0111] In an embodiment of the method for processing key information of project R & D materials in this application, the following content may also be specifically included: Step S701: Perform hierarchical clustering analysis on the R & D key information, calculate the semantic distance between information, construct a hierarchical clustering tree based on the semantic distance, map the hierarchical structure of the hierarchical clustering tree to the hierarchical structure of the document framework, set the information display priority according to the weight value of information in each level, and convert the hierarchical structure into a multi-level document framework based on the preset document organization rules, and set information filling position identifiers in the multi-level document framework; Step S702: Fill the information in the structured R & D document into the corresponding information filling positions in the order of decreasing weight value, use the text similarity algorithm to detect the information redundancy in the filled document, merge and streamline the redundant information based on the preset rules, construct a document verification model to verify the structural integrity and content consistency of the document, optimize and adjust the document according to the verification results, and output the adjusted document as the project application document.

[0112] Optionally, this embodiment first designs a dedicated hierarchical clustering analysis mechanism. In view of the characteristics of R & D key information, an improved hierarchical clustering algorithm is adopted, and the multi-dimensional measurement method is used to calculate the semantic distance between information: D = w1·cosine_dist + w2·edit_dist + w3·context_dist, where w1, w2, and w3 are weight coefficients, corresponding to cosine distance, edit distance, and context correlation distance respectively. This composite distance measurement can more accurately reflect the semantic association between R & D information.

[0113] This embodiment implements an innovative hierarchical clustering tree construction method. Adopting a bottom-up aggregation strategy, initially each piece of information is regarded as an independent cluster, and then the closest clusters are iteratively merged. The merging process takes into account the topic relevance and logical dependency of the information, calculates the distance between clusters through the Ward minimum variance criterion, and ensures the rationality of the clustering result. For highly specialized technical content, the system will appropriately increase the merging threshold to maintain its independence.

[0114] This embodiment designs an intelligent framework conversion mechanism for hierarchical structure mapping. First, analyze the hierarchical characteristics of the clustering tree, including depth, width, and branch balance. Then, according to the organization requirements of the document framework, dynamically map the clustering levels to the document levels. The mapping process uses heuristic rules to ensure that key content such as technical routes, innovation points, and experimental verifications is presented at appropriate levels.

[0115] This embodiment innovatively implements an information weight calculation method. The weight value comprehensively considers the importance, novelty, and relevance of the information: weight = α·importance + β·novelty + γ·relevance, where α, β, and γ are dynamically adjusted coefficients. For core content such as key technical breakthroughs and innovation achievements, the system will assign higher weight values.

[0116] This embodiment designs a complete document organization rule system. The rules include multiple dimensions such as chapter structure specifications, content organization logic, and format requirements. Through the rule engine, the hierarchical structure is converted into a standardized document framework, and at the same time, information filling positions are reserved in the framework, and the attribute requirements of the positions are recorded through identifiers.

[0117] This embodiment implements an efficient information filling strategy. The information sorted by weight is filled into the corresponding positions in turn, and the filling process combines template matching and context adaptation. For content cited across chapters, the system will automatically establish citation relationships to ensure the coherence of the document. After filling, the content layout is optimized through local adjustment to improve readability.

[0118] This embodiment innovatively designs an information redundancy detection mechanism. An improved text similarity algorithm is adopted, including analysis at two levels: literal similarity and semantic similarity. The document content is scanned through the sliding window technique to identify repeated or approximate expressions. For the detected redundant content, the system will selectively retain or merge it according to the importance of the context.

[0119] In this embodiment, a multi-level quality inspection framework is implemented for document verification. First, the integrity of the document structure is verified, including the existence of necessary chapters and the rationality of the hierarchical relationship. Then, the consistency of the content is checked, including aspects such as term usage, data citation, and logical relationships. The verification process combines rule checking and machine learning to identify potential problem points.

[0120] In this embodiment, an intelligent document optimization mechanism is designed. Based on the verification results, the system will automatically generate optimization suggestions, including structural adjustment, content supplementation, and expression optimization. The optimization process adopts an iterative approach, and verification is performed again after each round of optimization until the quality requirements are met. For problems that cannot be automatically optimized, the system will mark them for manual review.

[0121] In the document output link of this embodiment, format standardization processing is achieved. According to the project application requirements, the layout format of the document is automatically adjusted, including unified settings of elements such as fonts, paragraphs, and charts. At the same time, the table of contents structure and index of the document are generated for easy reference and citation.

[0122] Through the above technological innovations, this embodiment effectively solves multiple key problems in the generation of project application documents: unreasonable content organization, unclear structural hierarchy, information redundancy, unstable quality, etc. In practical applications, this solution can accurately understand and organize R & D information, and generate high-quality application documents that meet the requirements. It is particularly suitable for the application work of technological innovation projects, significantly improving the efficiency and quality of document generation. The adaptive characteristics of this solution enable it to handle different types of project application requirements, with broad application value. By intelligently organizing and optimizing the key R & D information, the professionalism and standardization of the application documents are ensured, effectively improving the success rate of project applications.

[0123] In order to effectively solve the deficiencies of traditional technologies in multi-modal information processing and knowledge structure optimization, and significantly improve the efficiency of R & D data management and project application, this application provides an embodiment of a project R & D data key information processing device for implementing all or part of the content of the project R & D data key information processing method. See Figure 2 The project R & D data key information processing device specifically includes the following contents: The project data extraction module 10 is used to import enterprise R & D data into the R & D content recognition system, decompose the R & D data into multi-modal information through the R & D content recognition system, extract text content, voice content, and image content, perform semantic segmentation on the text content, perform speaker separation and semantic recognition on the voice content, perform scene understanding and text recognition on the image content, align the text content, voice content, and image content in the time series dimension, construct a multi-modal information fusion matrix, and generate a semantic association vector based on the multi-modal information fusion matrix; The knowledge graph construction module 20 is used to construct an adaptive R & D knowledge graph, input the semantic association vector into the R & D semantic analysis model. The R & D semantic analysis model performs semantic completion and error correction on the semantic association vector based on the context relationship, generates a normalized semantic unit, constructs the normalized semantic unit into a knowledge graph node, calculates the node weight based on the association degree between the knowledge graph nodes, dynamically adjusts the knowledge graph structure according to the node weight, optimizes the topological structure of the adaptive R & D knowledge graph through feedback iteration, and generates a structured R & D document with a traceability relationship. The project data processing module 30 is used to input the structured R & D document into the R & D document understanding model trained based on a deep neural network. The R & D document understanding model extracts the key R & D information according to the topological structure of the adaptive R & D knowledge graph, constructs a multi-level document framework based on the weight distribution of the key information, fills the information in the structured R & D document into the multi-level document framework according to the information weight, and performs document consistency verification to generate a project application document.

[0124] As can be seen from the above description, the key information processing device for project R & D data provided in the embodiments of the present application can realize unified processing and temporal alignment of text, speech, and image content through constructing an R & D content recognition system and an adaptive R & D knowledge graph, and through multi-modal information decomposition and fusion. Based on the R & D semantic analysis model, semantic completion and error correction are performed, the knowledge graph structure is dynamically constructed and optimized, and a structured document with a traceability relationship is generated. The system uses a deep neural network model to extract the key R & D information and constructs a multi-level document framework, realizing the intelligent conversion from R & D data to project application documents. This method effectively solves the deficiencies of traditional technologies in multi-modal information processing and knowledge structure optimization, and significantly improves the efficiency of R & D data management and project application.

[0125] At the hardware level, in order to effectively solve the deficiencies of traditional technologies in multi-modal information processing and knowledge structure optimization, and significantly improve the efficiency of R & D data management and project application, the embodiments of the present application provide an electronic device for implementing all or part of the content in the key information processing method for project R & D data. The electronic device specifically includes the following content: A processor, a memory, a communications interface, and a bus; wherein, the processor, the memory, and the communications interface complete communication with each other through the bus; the communications interface is used to implement information transmission between the key information processing device for project R & D materials and related devices such as a core business system, a user terminal, and a related database, etc.; this logic controller can be a desktop computer, a tablet computer, a mobile terminal, etc., and this embodiment is not limited thereto. In this embodiment, this logic controller can be implemented with reference to the embodiments of the method for processing key information of project R & D materials and the embodiments of the key information processing device for project R & D materials, the content of which is incorporated herein, and the repeated parts will not be elaborated.

[0126] It can be understood that the user terminal can include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device can include smart glasses, a smart watch, a smart bracelet, etc.

[0127] In practical applications, part of the method for processing key information of project R & D materials can be executed on the electronic device side as described above, or all operations can be completed in the client device. Specifically, it can be selected according to the processing capacity of the client device and the limitations of the user usage scenario, etc. This application does not make any limitations in this regard. If all operations are completed in the client device, the client device can also include a processor.

[0128] The above-mentioned client device can have a communication module (i.e., a communication unit), and can be communicatively connected to a remote server to realize data transmission with the server. The server can include a server on the task scheduling center side, and in other implementation scenarios, it can also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server can include a single computer device, or can include a server cluster composed of multiple servers, or a server structure of a distributed device.

[0129] Figure 3 This is a schematic block diagram of the system composition of the electronic device 9600 according to an embodiment of the present application. As Figure 3 shown, the electronic device 9600 can include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It should be noted that this Figure 3 is exemplary; other types of structures can also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0130] In one embodiment, the functions of the key information processing method for project R & D materials can be integrated into the central processing unit 9100. Among them, the central processing unit 9100 can be configured to perform the following controls: Step S101: Import enterprise R & D materials into the R & D content recognition system. Through the R & D content recognition system, perform multi-modal information decomposition on the R & D materials, extract text content, speech content, and image content, perform semantic segmentation on the text content, perform speaker separation and semantic recognition on the speech content, perform scene understanding and text recognition on the image content, align the text content, speech content, and image content in the time series dimension, construct a multi-modal information fusion matrix, and generate a semantic association vector based on the multi-modal information fusion matrix; Step S102: Construct an adaptive R & D knowledge graph. Input the semantic association vector into the R & D semantic analysis model. The R & D semantic analysis model performs semantic completion and error correction on the semantic association vector based on the context relationship, generates a normalized semantic unit, constructs the normalized semantic unit as a knowledge graph node, calculates the node weight based on the association degree between the knowledge graph nodes, dynamically adjusts the knowledge graph structure according to the node weight, and optimizes the topological structure of the adaptive R & D knowledge graph through feedback iteration to generate a structured R & D document with a traceability relationship; Step S103: Input the structured R & D document into the R & D document understanding model trained based on a deep neural network. The R & D document understanding model extracts the key R & D information according to the topological structure of the adaptive R & D knowledge graph, constructs a multi-level document framework based on the weight distribution of the key information, fills the information in the structured R & D document into the multi-level document framework according to the information weight, and performs document consistency verification to generate a project application document.

[0131] As can be seen from the above description, the electronic device provided by the embodiment of the present application constructs an R & D content recognition system and an adaptive R & D knowledge graph. Through multi-modal information decomposition and fusion, it realizes the unified processing and time series alignment of text, speech, and image content. Based on the R & D semantic analysis model, it performs semantic completion and error correction, dynamically constructs and optimizes the knowledge graph structure, and generates a structured document with a traceability relationship. The system uses a deep neural network model to extract the key R & D information and constructs a multi-level document framework, realizing the intelligent conversion from R & D materials to project application documents. This method effectively solves the deficiencies of traditional technologies in multi-modal information processing and knowledge structure optimization, and significantly improves the efficiency of R & D material management and project application.

[0132] In another embodiment, the key information processing device for project R & D materials can be separately configured from the central processing unit 9100. For example, the key information processing device for project R & D materials can be configured as a chip connected to the central processing unit 9100, and the functions of the key information processing method for project R & D materials can be realized through the control of the central processing unit.

[0133] As Figure 3 shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It should be noted that the electronic device 9600 does not necessarily have to include Figure 3 all the components shown in Figure 3 ; in addition, the electronic device 9600 may further include Figure 3 components not shown in

[0134] As Figure 3 shown, the central processing unit 9100, sometimes also called a controller or operation control, may include a microprocessor or other processor devices and / or logic devices. The central processing unit 9100 receives inputs and controls the operations of the various components of the electronic device 9600.

[0135] Among them, the memory 9140, for example, may be one or more of a buffer, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory, or other suitable devices. The above information related to failures can be stored, and in addition, programs for executing relevant information can also be stored. And the central processing unit 9100 can execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0136] The input unit 9120 provides inputs to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to supply power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0137] The memory 9140 may be a solid-state memory. For example, a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be such a memory that stores information even when powered off, can be selectively erased and has more data stored. Examples of such a memory are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, and the application / function storage unit 9142 is used to store application programs and function programs or the processes for operating the electronic device 9600 through the central processing unit 9100.

[0138] The memory 9140 may further include a data storage unit 9143 for storing data such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for the communication functions of the electronic device and / or for performing other functions of the electronic device (such as a messaging application, an address book application, etc.).

[0139] The communication module 9110 is a transmitter / receiver that transmits and receives signals via the antenna 9111. The communication module 9110 (transmitter / receiver) is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as in the case of a conventional mobile communication terminal.

[0140] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module, and / or a wireless local area network module, etc. The communication module 9110 (transmitter / receiver) is also coupled to the speaker 9131 and the microphone 9132 via the audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby implementing normal telecommunication functions. The audio processor 9130 may include any suitable buffers, decoders, amplifiers, etc. Additionally, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.

[0141] Embodiments of the present application also provide a computer-readable storage medium capable of implementing all steps in the key information processing method for project R & D materials with the execution subject being a server or a client in the above embodiments. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, all steps in the key information processing method for project R & D materials with the execution subject being a server or a client in the above embodiments are implemented. For example, when the processor executes the computer program, the following steps are implemented: Step S101: Import enterprise R & D materials into the R & D content recognition system, perform multi-modal information decomposition on the R & D materials through the R & D content recognition system, extract text content, speech content, and image content, perform semantic segmentation on the text content, perform speaker separation and semantic recognition on the speech content, perform scene understanding and text recognition on the image content, align the text content, speech content, and image content in the time series dimension, construct a multi-modal information fusion matrix, and generate a semantic association vector based on the multi-modal information fusion matrix; Step S102: Construct an adaptive R & D knowledge graph, input the semantic association vector into the R & D semantic analysis model. The R & D semantic analysis model performs semantic completion and error correction on the semantic association vector based on the context relationship, generates a normalized semantic unit, constructs the normalized semantic unit as a knowledge graph node, calculates the node weight based on the association degree between the knowledge graph nodes, dynamically adjusts the knowledge graph structure according to the node weight, and optimizes the topological structure of the adaptive R & D knowledge graph through feedback iteration to generate a structured R & D document with a traceability relationship; Step S103: Input the structured R & D document into the R & D document understanding model trained based on a deep neural network. The R & D document understanding model extracts the key R & D information according to the topological structure of the adaptive R & D knowledge graph, constructs a multi-level document framework based on the weight distribution of the key information, fills the information in the structured R & D document into the multi-level document framework according to the information weight, and performs document consistency verification to generate a project application document.

[0142] As can be seen from the above description, the computer-readable storage medium provided by the embodiments of the present application constructs a R & D content recognition system and an adaptive R & D knowledge graph. Through multi-modal information decomposition and fusion, it realizes the unified processing and temporal alignment of text, voice, and image content. Based on the R & D semantic analysis model, semantic completion and error correction are performed, the knowledge graph structure is dynamically constructed and optimized, and a structured document with a traceability relationship is generated. The system uses a deep neural network model to extract key R & D information and constructs a multi-level document framework, realizing the intelligent conversion of R & D materials into project application documents. This method effectively solves the deficiencies of traditional technologies in multi-modal information processing and knowledge structure optimization, and significantly improves the efficiency of R & D material management and project application.

[0143] The embodiments of the present application also provide a computer program product that can implement all the steps in the key information processing method of project R & D materials with the execution subject being a server or a client in the above embodiments. When the computer program / instructions are executed by a processor, the steps of the key information processing method of project R & D materials are implemented. For example, the computer program / instructions implement the following steps: Step S101: Import enterprise R & D materials into the R & D content recognition system. The R & D content recognition system performs multi-modal information decomposition on the R & D materials, extracts text content, voice content, and image content, performs semantic segmentation on the text content, speaker separation and semantic recognition on the voice content, performs scene understanding and text recognition on the image content, aligns the text content, voice content, and image content in the time dimension, constructs a multi-modal information fusion matrix, and generates a semantic association vector based on the multi-modal information fusion matrix; Step S102: Construct an adaptive R & D knowledge graph, input the semantic association vector into the R & D semantic analysis model. The R & D semantic analysis model performs semantic completion and error correction on the semantic association vector based on the context relationship, generates a normalized semantic unit, constructs the normalized semantic unit into a knowledge graph node, calculates the node weight based on the association degree between the knowledge graph nodes, dynamically adjusts the knowledge graph structure according to the node weight, and optimizes the topological structure of the adaptive R & D knowledge graph through feedback iteration to generate a structured R & D document with a traceability relationship. Step S103: Input the structured R & D document into the R & D document understanding model trained based on a deep neural network. The R & D document understanding model extracts the key R & D information according to the topological structure of the adaptive R & D knowledge graph, constructs a multi-level document framework based on the weight distribution of the key information, fills the information in the structured R & D document into the multi-level document framework according to the information weight, and performs document consistency verification to generate a project application document.

[0144] As can be seen from the above description, the computer program product provided by the embodiments of the present application constructs an R & D content recognition system and an adaptive R & D knowledge graph. Through multi-modal information decomposition and fusion, it realizes the unified processing and temporal alignment of text, voice, and image content. Based on the R & D semantic analysis model, semantic completion and error correction are performed, the knowledge graph structure is dynamically constructed and optimized, and a structured document with a traceability relationship is generated. The system uses a deep neural network model to extract key R & D information and constructs a multi-level document framework, realizing the intelligent conversion of R & D materials into project application documents. This method effectively solves the deficiencies of traditional technologies in multi-modal information processing and knowledge structure optimization, and significantly improves the efficiency of R & D material management and project application.

[0145] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a device, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0146] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (devices), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices produce a means for implementing the functions specified in the flow Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0147] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction means that implements the functions specified in the flow Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the flow Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0149] Specific embodiments are applied in the present invention to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A method for processing key information of project R&D materials, characterized in that: The method comprises: Importing enterprise R&D data into a R&D content recognition system, performing multimodal information decomposition on the R&D data through the R&D content recognition system, extracting text content, voice content and image content, performing semantic segmentation on the text content, performing speaker separation and semantic recognition on the voice content, performing scene understanding and text recognition on the image content, aligning the text content, voice content and image content in a temporal dimension, constructing a multimodal information fusion matrix, and generating a semantic association vector based on the multimodal information fusion matrix; Construct an adaptive R&D knowledge graph, input the semantic association vector into the R&D semantic analysis model, the R&D semantic analysis model performs semantic completion and error correction on the semantic association vector based on the contextual relationship, generates a standardized semantic unit, constructs the standardized semantic unit into a knowledge graph node, calculates the node weight based on the degree of association between the knowledge graph nodes, dynamically adjusts the knowledge graph structure according to the node weight, optimizes the topological structure of the adaptive R&D knowledge graph through feedback iteration, and generates a structured R&D document with a traceability relationship; The structured R&D document is input into a R&D document understanding model trained based on a deep neural network. The R&D document understanding model extracts key R&D information according to the topological structure of the adaptive R&D knowledge graph, builds a multi-level document framework based on the weight distribution of the key information, fills the information in the structured R&D document into the multi-level document framework according to the information weight, performs document consistency verification, and generates a project application document.

2. The method for processing key information of project R&D materials according to claim 1, characterized in that: The step of importing the enterprise R&D data into the R&D content recognition system, performing multimodal information decomposition on the R&D data through the R&D content recognition system, extracting text content, voice content and image content, performing semantic segmentation on the text content, performing speaker separation and semantic recognition on the voice content, and performing scene understanding and text recognition on the image content includes: Classify the digital format research and development data according to the file format type and store them in a preset file directory; read the file content in the preset file directory based on a file parser; parse the file content according to the data structure to generate an initial data stream; divide the initial data stream into a text data stream, an audio data stream and an image data stream through the research and development content recognition system; perform text encoding conversion and format standardization on the text data stream; perform audio noise reduction and waveform normalization on the audio data stream; and perform image denoising and size standardization on the image data stream; Based on the semantic segmentation model, the text data stream is divided into independent semantic segments, the voiceprint features of the audio data stream are extracted and the speaker voiceprint library is constructed, the audio data stream is transcribed into text content using a speech recognition model, the image data stream is subjected to scene classification and target detection based on a convolutional neural network model, the text contained in the image is converted into text format using a text recognition model, and the semantic segments, transcribed text and recognized text are merged to construct a unified text corpus.

3. The method for processing key information of project R&D materials according to claim 1, characterized in that: The step of aligning the text content, the voice content, and the image content in a temporal dimension, constructing a multimodal information fusion matrix, and generating a semantic association vector based on the multimodal information fusion matrix includes: Based on the timestamp, the text content, voice content and image content are time-sequentially sorted, the time interval between adjacent contents is calculated, the time sequence features of each modal content are extracted using a bidirectional long short-term memory network, a time sequence feature vector is constructed, the time sequence feature vector is normalized, and the time sequence alignment between different modal contents is calculated based on a dynamic time warping algorithm, and the content with a time sequence alignment higher than a preset threshold is marked as an associated segment; The associated fragments are mapped to the feature space to construct a feature matrix, the attention mechanism is used to calculate the association weights between different modal features, the feature matrix is ​​weighted fused based on the association weights to obtain a multimodal information fusion matrix, the tensor decomposition method is used to reduce the dimension of the multimodal information fusion matrix, and the main feature components are extracted to construct a semantic association vector.

4. The method for processing key information of project R&D materials according to claim 1, characterized in that: The adaptive R&D knowledge graph is constructed, the semantic association vector is input into the R&D semantic analysis model, and the R&D semantic analysis model performs semantic completion and error correction on the semantic association vector based on the context relationship to generate a standardized semantic unit, including: Construct an ontology library in the research and development field, use the concept nodes, relationship types and attribute constraints in the ontology library as the basic framework of the knowledge graph, map the semantic association vector to the graph space based on the knowledge graph construction rules, use the named entity recognition model to identify the entity type from the semantic association vector, use the relationship extraction model to extract the semantic relationship between entities, and integrate the identified entities and relationships into the initial knowledge graph; A bidirectional Transformer model is used to contextually encode the semantic association vector, and the semantic relevance between word units is calculated based on the attention score. The missing semantic information in the semantic association vector is completed, and a semantic error correction model is constructed to correct ambiguous semantic expressions. The completed and corrected semantic information is constructed into a standardized semantic unit according to preset rules.

5. The method for processing key information of project R&D materials according to claim 1, characterized in that: The method of constructing the standardized semantic units into knowledge graph nodes, calculating node weights based on the degree of association between the knowledge graph nodes, dynamically adjusting the knowledge graph structure according to the node weights, optimizing the topological structure of the adaptive R&D knowledge graph through feedback iteration, and generating structured R&D documents with traceability relationships includes: Mapping the normalized semantic units into knowledge graph nodes according to the node construction rules, extracting feature vectors from the knowledge graph nodes, calculating the semantic similarity between nodes using the graph attention network, constructing a node adjacency matrix, calculating the node importance based on the node degree centrality and the feature vector centrality, combining the node importance with the semantic similarity to obtain a node weight value, and using the node weight value as the edge weight of the knowledge graph; A graph optimization objective function is constructed based on the edge weights, and the topological structure of the knowledge graph is iteratively optimized using the gradient descent method. The connectivity and coverage of the knowledge graph are evaluated in each round of iteration, and a stable knowledge graph structure is obtained when the optimization objective function converges. The node information is hierarchically organized based on the knowledge graph structure, and the organized node information is converted into a structured R&D document according to a preset template.

6. The method for processing key information of project R&D materials according to claim 1, characterized in that: The step of inputting the structured R&D document into a R&D document understanding model trained based on a deep neural network, wherein the R&D document understanding model extracts key R&D information according to the topological structure of the adaptive R&D knowledge graph, includes: Construct a multi-layer perceptron neural network as the basic architecture of the document understanding model, convert the structured R&D document into a word vector sequence and input it into the neural network, set a convolution layer in the neural network to extract local features, set a pooling layer to reduce the feature dimension, set a fully connected layer to fuse feature information, use a batch normalization method to standardize feature data, introduce nonlinear transformation through an activation function, and obtain a deep feature representation of the document; The topological structure of the adaptive R&D knowledge graph is converted into a graph structure feature vector, and the graph structure feature vector is extracted using a graph convolutional network. The extracted features are fused with the deep features of the document, and the correlation between the fused features and the key R&D information is calculated based on the attention mechanism. Features with a correlation higher than a preset threshold are selected as the key R&D information.

7. The method for processing key information of project R&D materials according to claim 1, characterized in that: The step of constructing a multi-level document framework based on the weight distribution of the key information, filling the information in the structured R&D document into the multi-level document framework according to the information weight, performing document consistency verification, and generating a project application document includes: Performing hierarchical clustering analysis on the key R&D information, calculating the semantic distance between the information, constructing a hierarchical clustering tree based on the semantic distance, mapping the hierarchical structure of the hierarchical clustering tree to the hierarchical structure of the document framework, setting the information display priority according to the weight value of the information in each hierarchy, converting the hierarchical structure into a multi-level document framework based on a preset document organization rule, and setting an information filling position identifier in the multi-level document framework; The information in the structured R&D document is filled into the corresponding information filling position in order of weight value from high to low, and the information redundancy in the filled document is detected by text similarity algorithm. The redundant information is merged and simplified based on preset rules, and a document verification model is constructed to verify the document structure integrity and content consistency. The document is optimized and adjusted according to the verification results, and the adjusted document is output as a project application document.

8. A device for processing key information of project R&D materials, characterized in that: The device comprises: A project data extraction module is used to import enterprise R&D data into the R&D content recognition system, perform multimodal information decomposition on the R&D data through the R&D content recognition system, extract text content, voice content and image content, perform semantic segmentation on the text content, perform speaker separation and semantic recognition on the voice content, perform scene understanding and text recognition on the image content, align the text content, voice content and image content in the time series dimension, construct a multimodal information fusion matrix, and generate a semantic association vector based on the multimodal information fusion matrix; A knowledge graph construction module is used to construct an adaptive R&D knowledge graph, input the semantic association vector into the R&D semantic analysis model, the R&D semantic analysis model performs semantic completion and error correction on the semantic association vector based on the contextual relationship, generates a standardized semantic unit, constructs the standardized semantic unit into a knowledge graph node, calculates the node weight based on the degree of association between the knowledge graph nodes, dynamically adjusts the knowledge graph structure according to the node weight, optimizes the topological structure of the adaptive R&D knowledge graph through feedback iteration, and generates a structured R&D document with a traceability relationship; The project data processing module is used to input the structured R&D document into the R&D document understanding model trained based on a deep neural network. The R&D document understanding model extracts key R&D information according to the topological structure of the adaptive R&D knowledge graph, builds a multi-level document framework based on the weight distribution of the key information, fills the information in the structured R&D document into the multi-level document framework according to the information weight, performs document consistency verification, and generates a project application document.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method for processing key information of project R&D materials described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for processing key information of project research and development materials described in any one of claims 1 to 7 are implemented.

Images