Open source software characteristic identification method, system and device based on semantic recall and prior knowledge verification and medium

Abstract

The application discloses an open source software feature identification method, system, device and medium based on semantic recall and prior knowledge verification, comprising the following steps: extracting text files of an open source project repository and preprocessing to generate a document object list; converting the document object list into a vector representation by using a pre-trained language model, and screening standard features higher than a set threshold through cosine similarity calculation to form a candidate feature set; sequentially performing keyword matching, regular expression rule matching and context conflict detection to generate a verification result containing verification status and evidence information; screening and fusing the candidate feature set by using a first arbitration rule to obtain a final feature list, and generating a structured report containing feature information, evidence chain and arbitration reasoning. The application realizes the collaborative optimization of high recall rate and high accuracy in open source software feature identification by constructing a two-stage architecture integrating semantic recall and prior knowledge verification, and introducing a conflict perception arbitration mechanism.

Description

Technical Field

[0001] This invention relates to the field of feature recognition technology, and in particular to open-source software feature recognition methods, systems, devices and media based on semantic recall and prior knowledge verification. Background Technology

[0002] With the development of the global open-source software ecosystem, the software supply chain has become increasingly complex, posing unprecedented challenges to enterprises in selecting open-source software, conducting security audits, and managing compliance. Key feature information is often scattered across project README files, official documentation, code comments, and issue discussions, making information integration extremely difficult. Manual review methods are inefficient and difficult to scale. Against this backdrop, software feature identification technology has emerged. The core objective of software feature identification is to automatically and accurately extract information crucial for software management and risk control from various types of unstructured text.

[0003] However, current main software feature identification methods have significant limitations and cannot meet the large-scale application requirements for high-precision identification. While keyword matching methods are simple to implement and can achieve high accuracy in perfect matching scenarios, their recall is generally low due to their inability to effectively handle synonyms and semantic variations, resulting in the systematic omission of many real features. Rule engine-based methods exhibit extremely high accuracy when processing structured configuration files or dependency lists, but their performance drops significantly when dealing with unstructured natural language text. Furthermore, the construction and maintenance of rule bases require continuous expert investment, making them costly and difficult to adapt to the rapidly changing software ecosystem. Semantic methods based on deep learning, such as pre-trained language models, can convert text into vector representations, significantly improving coverage of diverse expressions. However, purely semantic methods suffer from semantic illusion, potentially making incorrect judgments based on semantic relevance. For example, they might misjudge text discussing the impact of vulnerabilities as actual component usage, or incorrectly identify unsupported features as supported by ignoring negative words. These problems are unacceptable in high-precision scenarios such as security auditing.

[0004] Furthermore, existing methods generally lack mechanisms for detecting and resolving contradictions in documents. For example, when there are inconsistencies between the project's declaration file and the code implementation, current technologies cannot effectively handle such conflicts. Therefore, there is an urgent need to develop an open-source software feature identification method based on semantic recall and prior knowledge verification. Summary of the Invention

[0005] In view of the aforementioned existing problems, the present invention is proposed.

[0006] Therefore, this invention provides an open-source software feature identification method, system, device, and medium based on semantic recall and prior knowledge verification, which solves the problem that existing methods cannot achieve both high recall and high precision with a single technical path, and lack an effective detection and arbitration mechanism for negative statements and contradictory information in documents, resulting in insufficient reliability of identification results.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, this invention provides an open-source software feature identification method based on semantic recall and prior knowledge verification, including: Extract text files from open-source project repositories and preprocess them to generate a structured list of document objects; The list of document objects is converted into a vector representation using a pre-trained language model, and standard features that exceed a set threshold are filtered by cosine similarity calculation to form a candidate feature set. Iterate through each candidate feature in the candidate feature set and sequentially perform keyword matching, regular expression rule matching, and context conflict detection to generate a verification result containing verification status and evidence information; Based on the verification results, the candidate feature set is filtered and fused using the first arbitration rule to obtain the final feature list; The final list of characteristics is sorted by confidence level to generate a structured report containing characteristic information, evidence chains, and arbitration reasoning.

[0008] As a preferred embodiment of the open-source software feature identification method based on semantic recall and prior knowledge verification described in this invention, the formation of the candidate feature set includes: Each sentence in the document object list is traversed, and a pre-trained sentence converter model is used to encode the sentence into a normalized vector representation. Based on the first feature embedding library, the K most similar standard features for each sentence vector are retrieved using an approximate nearest neighbor search algorithm; Retain the matching results that are higher than the preset similarity threshold among the K standard features; The candidate features retrieved from all sentences are aggregated and deduplicated, and the highest similarity score for each feature is retained to form a candidate feature set.

[0009] As a preferred embodiment of the open-source software feature identification method based on semantic recall and prior knowledge verification described in this invention, the construction of the first feature embedding library includes: Collect standard features to be identified, each of which includes a standard name, multiple synonyms, and a category label; Concatenate the standard name of each standard feature with all its synonyms into a continuous text string; The concatenated text string is encoded into a fixed-dimensional normalized vector using a pre-trained language model; The generated feature vectors and their metadata are stored in a vector database to construct an index structure that supports approximate nearest neighbor search.

[0010] As a preferred embodiment of the open-source software feature identification method based on semantic recall and prior knowledge verification described in this invention, the generation of verification results including verification status and evidence information includes: Obtain the feature identifier of each candidate feature and the text position matched in the document, and initialize the verification state to no evidence; Search for strong keywords corresponding to the candidate feature in the original document. If a match is found, set the verification status of the candidate feature to strong support. If no strong keyword is found, continue searching for weak keywords corresponding to the candidate feature. If a match is found, set the verification status of the candidate feature to weak support. The document is scanned using the regular expression rules corresponding to the candidate features. In response to any successful match of the regular expression rule, the verification status of the candidate feature is upgraded to strong support. Conflict detection is performed within the context window of the matched keywords or rule evidence. If an explicit negative word or conflict indicator word is detected, the verification status of the candidate feature is overwritten as conflict, and the conflict information is recorded. If no keywords or rules are matched, the verification status is set to No Evidence; The output includes the verification result, which contains verification status and evidence information.

[0011] As a preferred embodiment of the open-source software feature identification method based on semantic recall and prior knowledge verification described in this invention, the conflict detection includes: Extract the context text within a set range before and after the location of each keyword or rule-matching evidence; Search the context text for predefined explicit negation words and conflict indicators; If any negative word or conflict indicator is detected, it is determined that there is evidence of conflict. The verification status of the candidate feature is immediately overwritten as conflict, and the conflict information is recorded.

[0012] As a preferred embodiment of the open-source software feature identification method based on semantic recall and prior knowledge verification described in this invention, the step of filtering and fusing the candidate feature set using the first arbitration rule includes: Obtain the semantic similarity score and verification status of the candidate features; If the verification status is a conflict, the candidate feature is rejected directly. If the verification status is strong support, the candidate feature is confirmed as the final output, and confidence fusion is performed based on the semantic similarity score and knowledge weight. If the verification status is weak support and the semantic similarity score is higher than the first threshold, then the candidate feature is confirmed with low confidence and a weighted fusion calculation is performed. If the verification status is no evidence and the semantic similarity score is higher than the second threshold, the candidate feature is confirmed with low confidence. In all other cases, the candidate feature is rejected. All confirmed candidate features are sorted in descending order of final confidence level to generate a final feature list.

[0013] As a preferred embodiment of the open-source software feature identification method based on semantic recall and prior knowledge verification described in this invention, the generated structured document object list includes: Pull code and related documentation from open-source project repositories, extracting text files including README files, official documentation, and code comments; The text file is cleaned and denoised. The cleaned text is segmented using boundary detection tools, and texts exceeding the model input limit are segmented using a sliding window method. Short sentences with fewer than a preset threshold of words are filtered out, the original text of each sentence and its position in the document are recorded, and a structured list of document objects is generated.

[0014] Secondly, this invention provides an open-source software feature recognition system based on semantic recall and prior knowledge verification, including: The data acquisition module is used to extract text files from open-source project repositories and preprocess them to generate a structured list of document objects. The semantic recall module is used to convert the list of document objects into vector representations using a pre-trained language model, and to filter standard features that are higher than a set threshold by cosine similarity calculation to form a candidate feature set. The knowledge verification module is used to traverse each candidate feature in the candidate feature set and sequentially perform keyword matching, regular expression rule matching, and context conflict detection to generate a verification result containing verification status and evidence information. The arbitration fusion module is used to filter and fuse the candidate feature set based on the verification result using the first arbitration rule to obtain the final feature list; The results output module is used to sort the final feature list by confidence level and generate a structured report containing feature information, evidence chain and arbitration reasoning.

[0015] Thirdly, the present invention provides an electronic device, including a memory and a processor; the memory is used to store computer-executable instructions, and the processor, when executing the computer-executable instructions, implements the steps of an open-source software feature identification method based on semantic recall and prior knowledge verification.

[0016] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of an open-source software feature identification method based on semantic recall and prior knowledge verification.

[0017] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention achieves synergistic optimization of high recall and high precision in open-source software feature identification by constructing a two-stage architecture that integrates semantic recall and prior knowledge verification, and introducing a conflict-aware arbitration mechanism. In the semantic recall stage, this invention fully utilizes the generalization ability of deep learning models to effectively cover synonyms and diverse expressions, ensuring that key features are not overlooked. In the prior knowledge verification stage, this invention conducts rigorous fact-checking on candidate features through precise keyword matching, regular rule detection, and contextual negation word recognition, effectively suppressing false positives caused by semantic illusions. In the arbitration fusion stage, this invention introduces a conflict veto rule, which can identify and eliminate contradictory information in documents, significantly improving the reliability and interpretability of the identification results. Compared with existing technologies, this invention can provide more accurate and reliable feature identification results in large-scale automated auditing scenarios, providing solid technical support for open-source software selection, security vulnerability investigation, and license compliance management. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the overall process logic of an open-source software feature identification method based on semantic recall and prior knowledge verification, provided as an embodiment of the present invention.

[0020] Figure 2 This is an overall architecture diagram of an open-source software feature identification method based on semantic recall and prior knowledge verification, provided as an embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram of the knowledge verification state transition of an open-source software feature identification method based on semantic recall and prior knowledge verification, provided as an embodiment of the present invention. Detailed Implementation

[0022] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0023] Example 1, referring to Figure 1 As one embodiment of the present invention, an open-source software feature identification method based on semantic recall and prior knowledge verification is provided, such as... Figure 1 The specific steps shown are as follows: S100: Extract text files from open-source project repositories and preprocess them to generate a structured list of document objects; S200: Use a pre-trained language model to convert a list of document objects into a vector representation, and use cosine similarity calculation to filter out standard features that are higher than a set threshold to form a candidate feature set; S300: Traverse each candidate feature in the candidate feature set and sequentially perform keyword matching, regular expression rule matching, and context conflict detection to generate a verification result containing verification status and evidence information; S400: Based on the verification results, the candidate feature set is filtered and fused using the first arbitration rule to obtain the final feature list; S500: Sort the final feature list by confidence level and generate a structured report containing feature information, chain of evidence, and arbitration reasoning.

[0024] It should be noted that this invention achieves synergistic optimization of high recall and high precision in open-source software feature identification by constructing a two-stage architecture that integrates semantic recall and prior knowledge verification, and by introducing a conflict-aware arbitration mechanism. In the semantic recall stage, this invention fully utilizes the generalization ability of deep learning models to effectively cover synonyms and diverse expressions, ensuring that key features are not overlooked. In the prior knowledge verification stage, this invention conducts rigorous fact-checking on candidate features through precise keyword matching, regular rule detection, and contextual negation word recognition, effectively suppressing false positives caused by semantic illusions. In the arbitration fusion stage, this invention introduces a conflict veto rule, which can identify and eliminate contradictory information in documents, significantly improving the reliability and interpretability of the identification results.

[0025] Example 2, refer to Figure 2 and Figure 3 Based on the previous embodiment, this embodiment provides a specific implementation of an open-source software feature identification method based on semantic recall and prior knowledge verification, in order to illustrate the technical means used in this method.

[0026] Step S100 extracts and preprocesses text files from open-source project repositories to generate a structured list of document objects, including the following sub-steps A1 to A4: In A1: Pull code and related documentation from open-source project repositories, and extract text files including README files, official documentation, and code comments; In A2: Clean and denoise the text file; Specifically, it filters out HTML tags, Markdown formatting symbols, and other content in text files that is irrelevant to the semantics of natural language.

[0027] In A3: Boundary detection tools are used to segment the cleaned text into sentences, and sliding window methods are used to segment texts whose length exceeds the model input limit; Specifically, tools such as NLTK or spaCy are used for sentence boundary detection to segment the entire document into independent sentences. For long documents exceeding the model's maximum input limit, a sliding window approach is used for segmentation, with a 20% overlap to avoid segmenting key information at window boundaries. In this embodiment, the model's maximum input limit is 512 tokens.

[0028] In A4: Filter out short sentences with fewer than a preset threshold of words, record the original text of each sentence and the sentence's position in the document, and generate a structured list of document objects; Specifically, the preset threshold can be flexibly set according to the actual application scenario and the characteristics of the corpus. Sentences with fewer than five words are usually considered too short and filtered out, because such sentences often only contain title words or incomplete expressions and are difficult to carry substantial software feature information. For texts that are mainly code comments, the threshold can be appropriately lowered to retain effective information in abbreviated form, while for the main body of the document, the threshold can be appropriately raised to remove noise.

[0029] It should be noted that step S100 transforms unstructured raw information into standardized processing units, laying a complete and orderly data foundation for subsequent accurate identification and ensuring that key characteristic information is not missed due to messy format or scattered location.

[0030] Step S200 uses a pre-trained language model to convert the list of document objects into vector representations, and uses cosine similarity calculation to filter standard features that exceed a set threshold, forming a candidate feature set including: In this embodiment of the invention, forming a candidate feature set includes: Iterate through each sentence in the list of document objects and encode the sentence into a normalized vector representation using a pre-trained sentence converter model; Based on the first feature embedding library, the K most similar standard features for each sentence vector are retrieved using an approximate nearest neighbor search algorithm; Retain the matching results that are higher than the preset similarity threshold among the K standard features; The candidate features retrieved from all sentences are aggregated and deduplicated, and the highest similarity score for each feature is retained to form a candidate feature set.

[0031] Specifically, for each sentence vector, the FAISS index is used to retrieve the top-K most similar features from the feature embedding library. A preset similarity threshold is set, and only features with similarity scores higher than this threshold are retained as candidate features.

[0032] For example, the FAISS index is used to retrieve the 10 most similar features in the feature embedding library, and a preset similarity threshold of 0.7 is set. Features with similarity scores higher than 0.7 are retained.

[0033] Specifically, candidate features retrieved from all sentences are aggregated. For cases where the same feature is retrieved multiple times in different sentences, deduplication is performed, retaining only the match with the highest similarity for that feature. Finally, all candidate features are sorted in descending order of their semantic similarity scores to form a candidate feature set.

[0034] It should be noted that, in order to improve performance, batch encoding (processing 32-64 sentences at a time) can be used to improve GPU utilization; for ultra-large feature libraries, index sharding and parallel retrieval can be performed; an early stopping mechanism can also be implemented, which skips subsequent texts in advance when a certain number of sentences are processed without producing a high-scoring match.

[0035] In an optional embodiment, the candidate feature set can also be obtained based on a hybrid retrieval method that combines dense and sparse retrieval. While generating semantic vectors using a pre-trained language model, traditional word frequency statistics methods such as BM25 are introduced to construct sparse vectors. By weighting and fusing or reordering the two types of retrieval results, the sensitivity to precise term matching can be enhanced while retaining semantic generalization ability.

[0036] In an optional embodiment, the acquisition of the candidate feature set can also adopt a two-stage retrieval architecture based on a deep learning ranking model. The first stage uses a lightweight dual encoder model to quickly recall a large number of candidates, while the second stage introduces an interactive cross-encoder model to refine and re-rank the recall results.

[0037] In this embodiment of the invention, the construction of the first feature embedding library includes: Collect the standard features to be identified. Each standard feature includes a standard name, multiple synonyms, and a category label. Concatenate the standard name of each standard feature with all its synonyms into a continuous text string; The concatenated text string is encoded into a fixed-dimensional normalized vector using a pre-trained language model; The generated feature vectors and their metadata are stored in a vector database to build an index structure that supports approximate nearest neighbor search.

[0038] Specifically, collect and organize N standard feature descriptions to be identified. Each standard feature includes its standard name, such as "using the Log4j logging component"; multiple synonyms, such as ["Apache Log4j", "log4j-core", "log4j2"]; and related category tags, such as [dependency component, logging framework]. Iterate through each standard feature, and for each standard feature, concatenate its standard name and all its synonyms into a continuous text string.

[0039] Specifically, loading a pre-trained sentence converter model, such as 'sentence-transformers / all-mpnet-base-v2', and choosing a sentence converter model that has been fine-tuned on code or technical corpora will yield better results.

[0040] Specifically, the loaded model encodes the concatenated text string into a fixed-dimensional vector representation, such as 768-dimensional vectors, and then normalizes the vectors. Normalization ensures that subsequent cosine similarity calculations are equivalent to vector inner product calculations, thereby accelerating computation using optimized inner product retrieval. The generated feature vectors and their metadata are stored in a vector database, and an efficient index structure is built using a vector database such as FAISS. This index supports approximate nearest neighbor search and can achieve millisecond-level retrieval in a feature database of millions of elements.

[0041] It should be noted that step S200 above utilizes the semantic understanding capability of the pre-trained language model to convert the text into a high-dimensional vector and calculate the similarity with the standard feature library. This can effectively overcome the limitations of synonym matching in keyword matching, identify diverse expressions, and generate a preliminary candidate set while ensuring a high recall rate.

[0042] Step S300 iterates through each candidate feature in the candidate feature set and sequentially performs keyword matching, regular expression rule matching, and context conflict detection to generate a verification result containing verification status and evidence information, including: In this embodiment of the invention, the prior knowledge base is designed with a three-layer structure, the details of which include: The first layer is a precise keyword dictionary. This dictionary defines two types of keywords for each feature: "strong keywords," which refer to words or phrases that are very specific and strongly suggest the existence of the feature, such as the specific package name 'org.apache.log4j'; and "weak keywords," which refer to related but less specific words, such as the general technical term 'logging utility'. A context window size is also set for keyword matching, such as 50 characters before and after.

[0043] The second layer is the regular expression rule library. High-precision regular expression rules are defined for structured or semi-structured content such as code files and configuration files. It should be noted that regular expression rules are usually associated with specific file types, such as '.xml' or '.java', and assigned a high confidence level.

[0044] The third layer is a conflict / negation knowledge base. This base contains two types of patterns: first, "explicit negation words," such as 'do not support', 'not supported', 'disabled', etc.; second, "conflict indicator words," such as 'vulnerability in', 'considering', 'plan to', etc., which suggest that the current text does not affirmatively describe the feature. Similarly, a context window is set for negation detection.

[0045] In embodiments of the present invention, such as Figure 3 As shown, the generated verification result, which includes verification status and evidence information, includes: Obtain the feature identifier of each candidate feature and the text position matched in the document, and initialize the verification state to no evidence; Search for strong keywords corresponding to candidate features in the original document. If a match is found, set the verification status of the candidate feature to strong support. If no strong keyword is found, continue searching for weak keywords corresponding to the candidate feature. If a match is found, set the verification status of the candidate feature to weak support. The document is scanned using the regular expression rules corresponding to the candidate features. In response to a successful match of any regular expression rule, the validation status of the candidate feature is upgraded to strong support. Conflict detection is performed within the context window of the matched keywords or rule evidence. If an explicit negative word or conflict indicator word is detected, the verification status of the candidate feature is overwritten as conflict and the conflict information is recorded. If no keywords or rules are matched, the verification status is set to No Evidence; The output includes the verification result, which contains verification status and evidence information.

[0046] In an optional embodiment, code structure parsing and dependency graph analysis techniques can also be introduced. By parsing the import statements in the project's build files or source code, a dependency graph and component call relationships of the project can be constructed, and candidate features can be cross-validated with actual code dependencies.

[0047] In an optional embodiment, fine-grained text analysis techniques based on named entity recognition and relation extraction can also be used. A trained deep learning model is used to automatically identify entities from the document and extract their semantic relationships with the main body of the project. The structured knowledge extraction results are then matched and verified with candidate features.

[0048] Specifically, the four mutual exclusion verification states are shown in Table 1.

[0049] Table 1: Verification Status.

[0050] In this embodiment of the invention, the collision detection step includes: Extract the context text within a set range before and after the location of each keyword or rule-matching evidence; Search the context text for predefined explicit negation words and conflict indicators; If any negative word or conflict indicator is detected, it is determined that there is evidence of conflict. The verification status of the candidate feature is immediately overwritten as conflict, and the conflict information is recorded.

[0051] It should be noted that step S300 above introduces a hierarchical verification mechanism. First, it uses precise keywords and regular rules to perform fact-checking on candidate features. Then, it detects negative words and conflict indicators in the context window. This can effectively identify negative statements and contradictory information between documents and code, suppress false alarms caused by semantic illusions from the root, and generate a traceable chain of evidence for each identification result.

[0052] Step S400, based on the verification results, uses the first arbitration rule to filter and fuse the candidate feature set, resulting in the final feature list, which includes: In this embodiment of the invention, the screening and fusion of the candidate feature set using the first arbitration rule includes: Obtain the semantic similarity score and verification status of candidate features; If the verification status is conflicting, the candidate feature is rejected directly because there is clear evidence of conflict, and it is not credible no matter how high the semantic score is. If the verification status is strong support, the candidate feature is confirmed as the final output, and confidence is fused based on semantic similarity score and knowledge weight; If the verification status is weak support and the semantic similarity score is higher than the first threshold, then the candidate feature is confirmed with low confidence and weighted fusion calculation is performed. If the verification status is no evidence and the semantic similarity score is higher than the second threshold, the candidate feature is confirmed with low confidence. In all other cases, candidate features are rejected. All confirmed candidate features are sorted in descending order of final confidence level to generate a final feature list.

[0053] It should be noted that the values ​​of the first and second thresholds can be dynamically adjusted according to the different emphases on precision and recall in the actual application scenario. Typically, the first threshold is set to 0.85 in the standard configuration to require a higher semantic similarity in the case of weak support to compensate for the lack of evidence strength, and the second threshold is set to 0.92 to set extremely strict admission criteria in the case of no evidence to control the false positive risk of pure semantic recall.

[0054] Specifically, the mathematical expression for the arbitration logic is: in, This represents the semantic similarity score. The calculation rules are shown in Table 2: Table 2: Calculation Rules Table.

[0055] In this embodiment of the invention, the system can perform batch arbitration on multiple candidate features in actual processing. The detailed steps include: Iterate through all candidate features that have undergone semantic recall and knowledge verification; The above core arbitration rules are applied to each candidate feature to obtain the decision result; Collect all characteristics of decisions that are "confirmed" or "low-confidence confirmation"; These finally confirmed characteristics are sorted in descending order according to their calculated final confidence levels; Output a sorted list of high-confidence features with detailed evidence and reasoning.

[0056] It should be noted that step S400 above organically integrates semantic similarity and knowledge verification status, which can retain effective recall results to the maximum extent while ensuring high accuracy, achieve synergistic optimization of recognition accuracy and coverage, and significantly improve the reliability of the final output.

[0057] Step S500 sorts the final feature list by confidence level and generates a structured report containing feature information, chain of evidence, and arbitration reasoning, including: Obtain each feature and its corresponding final confidence level after arbitration fusion, and sort the feature list in descending order of confidence level; Organize the complete evidence chain information for each feature, including the original sentence matched in the semantic recall stage and its similarity score, the keywords or regular rules triggered in the prior knowledge verification stage and their verification status, and the arbitration rules and reasoning process applied in the arbitration fusion stage. All information is encapsulated according to a preset data structure to generate a structured report containing feature name, confidence level, source of evidence, verification evidence, and arbitration reasoning chain. This report can be stored in JSON format for easy exchange between systems. Optionally, a visual version for human review can be generated to intuitively display the judgment basis and complete decision path for each feature.

[0058] It should be noted that step S500 above generates a standardized report containing a complete chain of evidence and reasoning process, making each identification result interpretable and traceable. This not only facilitates manual review and verification but also provides clear feedback for subsequent model iteration and optimization, improving the usability and credibility of the technical solution in real-world engineering scenarios.

[0059] Example 3: This example provides an open-source software feature recognition system based on semantic recall and prior knowledge verification, including: The data acquisition module is used to extract text files from open-source project repositories and preprocess them to generate a structured list of document objects. The semantic recall module is used to convert a list of document objects into a vector representation using a pre-trained language model, and to filter standard features that exceed a set threshold by calculating cosine similarity to form a candidate feature set. The knowledge verification module is used to traverse each candidate feature in the candidate feature set and sequentially perform keyword matching, regular expression rule matching, and context conflict detection to generate a verification result containing verification status and evidence information. The arbitration fusion module is used to filter and fuse the candidate feature set based on the verification results using the first arbitration rule to obtain the final feature list; The results output module sorts the final feature list by confidence level and generates a structured report containing feature information, evidence chain, and arbitration reasoning.

[0060] It should be noted that the technical solution of the open-source software feature identification system based on semantic recall and prior knowledge verification is based on the same concept as the technical solution of the open-source software feature identification method based on semantic recall and prior knowledge verification mentioned above. For details not described in detail in the technical solution of the open-source software feature identification system based on semantic recall and prior knowledge verification in this embodiment, please refer to the description of the technical solution of the open-source software feature identification method based on semantic recall and prior knowledge verification mentioned above.

[0061] The above-mentioned unit modules can be embedded in the processor of the electronic device in hardware form or independent of it, or they can be stored in the memory of the electronic device in software form, so that the processor can call and execute the corresponding operations of the above modules.

[0062] This embodiment also provides an electronic device, which includes a processor, a memory, a communication interface, a display screen, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an open-source software feature identification method based on semantic recall and prior knowledge verification. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0063] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method proposed in the above embodiments.

[0064] The storage medium proposed in this embodiment belongs to the same inventive concept as the method proposed in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0065] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory, random access memory, flash memory, hard disk, or optical disk, and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute the method of the embodiments of the present invention.

[0066] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the present invention.

Claims

1. An open-source software feature identification method based on semantic recall and prior knowledge verification, characterized in that, include: Extract text files from open-source project repositories and preprocess them to generate a structured list of document objects; The list of document objects is converted into a vector representation using a pre-trained language model, and standard features that exceed a set threshold are filtered by cosine similarity calculation to form a candidate feature set. Iterate through each candidate feature in the candidate feature set and sequentially perform keyword matching, regular expression rule matching, and context conflict detection to generate a verification result containing verification status and evidence information; Based on the verification results, the candidate feature set is filtered and fused using the first arbitration rule to obtain the final feature list; The final list of characteristics is sorted by confidence level to generate a structured report containing characteristic information, evidence chains, and arbitration reasoning.

2. The open-source software feature identification method based on semantic recall and prior knowledge verification as described in claim 1, characterized in that, The formation of the candidate feature set includes: Each sentence in the document object list is traversed, and a pre-trained sentence converter model is used to encode the sentence into a normalized vector representation. Based on the first feature embedding library, the K most similar standard features for each sentence vector are retrieved using an approximate nearest neighbor search algorithm; Retain the matching results that are higher than the preset similarity threshold among the K standard features; The candidate features retrieved from all sentences are aggregated and deduplicated, and the highest similarity score for each feature is retained to form a candidate feature set.

3. The open-source software feature identification method based on semantic recall and prior knowledge verification as described in claim 2, characterized in that, The construction of the first feature embedding library includes: Collect standard features to be identified, each of which includes a standard name, multiple synonyms, and a category label; Concatenate the standard name of each standard feature with all its synonyms into a continuous text string; The concatenated text string is encoded into a fixed-dimensional normalized vector using a pre-trained language model; The generated feature vectors and their metadata are stored in a vector database to construct an index structure that supports approximate nearest neighbor search.

4. The open-source software feature identification method based on semantic recall and prior knowledge verification as described in claim 2, characterized in that, The generation of a verification result containing verification status and evidence information includes: Obtain the feature identifier of each candidate feature and the text position matched in the document, and initialize the verification state to no evidence; Search for strong keywords corresponding to the candidate feature in the original document. If a match is found, set the verification status of the candidate feature to strong support. If no strong keyword is found, continue searching for weak keywords corresponding to the candidate feature. If a match is found, set the verification status of the candidate feature to weak support. The document is scanned using the regular expression rules corresponding to the candidate features. In response to any successful match of the regular expression rule, the verification status of the candidate feature is upgraded to strong support. Conflict detection is performed within the context window of the matched keywords or rule evidence. If an explicit negative word or conflict indicator word is detected, the verification status of the candidate feature is overwritten as conflict, and the conflict information is recorded. If no keywords or rules are matched, the verification status is set to No Evidence; The output includes the verification result, which contains verification status and evidence information.

5. The open-source software feature identification method based on semantic recall and prior knowledge verification as described in claim 4, characterized in that, The collision detection includes: Extract the context text within a set range before and after the location of each keyword or rule-matching evidence; Search the context text for predefined explicit negation words and conflict indicators; If any negative word or conflict indicator is detected, it is determined that there is evidence of conflict. The verification status of the candidate feature is immediately overwritten as conflict, and the conflict information is recorded.

6. The open-source software feature identification method based on semantic recall and prior knowledge verification as described in claim 5, characterized in that, The step of using the first arbitration rule to filter and fuse the candidate feature set includes: Obtain the semantic similarity score and verification status of the candidate features; If the verification status is a conflict, the candidate feature is rejected directly. If the verification status is strong support, the candidate feature is confirmed as the final output, and confidence fusion is performed based on the semantic similarity score and knowledge weight. If the verification status is weak support and the semantic similarity score is higher than the first threshold, then the candidate feature is confirmed with low confidence and a weighted fusion calculation is performed. If the verification status is no evidence and the semantic similarity score is higher than the second threshold, the candidate feature is confirmed with low confidence. In all other cases, the candidate feature is rejected. All confirmed candidate features are sorted in descending order of final confidence level to generate a final feature list.

7. The open-source software feature identification method based on semantic recall and prior knowledge verification as described in claim 1, characterized in that, The generated list of structured document objects includes: Pull code and related documentation from open-source project repositories, extracting text files including README files, official documentation, and code comments; The text file is cleaned and denoised. The cleaned text is segmented using boundary detection tools, and texts exceeding the model input limit are segmented using a sliding window method. Short sentences with fewer than a preset threshold of words are filtered out, the original text of each sentence and its position in the document are recorded, and a structured list of document objects is generated.

8. An open-source software feature identification system based on semantic recall and prior knowledge verification, employing the open-source software feature identification method based on semantic recall and prior knowledge verification as described in any one of claims 1 to 7, characterized in that, include: The data acquisition module is used to extract text files from open-source project repositories and preprocess them to generate a structured list of document objects. The semantic recall module is used to convert the list of document objects into vector representations using a pre-trained language model, and to filter standard features that are higher than a set threshold by cosine similarity calculation to form a candidate feature set. The knowledge verification module is used to traverse each candidate feature in the candidate feature set and sequentially perform keyword matching, regular expression rule matching, and context conflict detection to generate a verification result containing verification status and evidence information. The arbitration fusion module is used to filter and fuse the candidate feature set based on the verification result using the first arbitration rule to obtain the final feature list; The results output module is used to sort the final feature list by confidence level and generate a structured report containing feature information, evidence chain and arbitration reasoning.

9. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store computer-executable instructions, and when the processor executes the computer-executable instructions, it implements the steps of the open-source software feature identification method based on semantic recall and prior knowledge verification as described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer-executable instructions stored thereon, characterized in that: When the computer-executable instructions are executed by the processor, they implement the steps of the open-source software feature identification method based on semantic recall and prior knowledge verification as described in any one of claims 1 to 7.

Images