A method for determining the source of open source code for aviation airborne software
By extracting features and calculating similarity from open-source code repositories, constructing a keyword dictionary, and scoring based on key information, the problem of inaccurate source identification of open-source code is solved, achieving automated and efficient source identification.
Patent Information
- Application Number
- CN202411957367.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-27
- Publication Date
- 2025-10-28
- Estimated Expiration
- 2044-12-27
AI Technical Summary
Existing technologies make it difficult to accurately determine the source of open-source code, especially when there are frequent cross-references between open-source software. Traditional methods rely on creation time, which leads to inaccurate determinations.
By extracting features and calculating similarity from open-source codebases, a keyword dictionary is constructed, scoring is performed based on key information, and a combination of various meta-information is analyzed to determine the source of the open-source code.
It enables automatic identification of open-source code sources, improving identification efficiency, accuracy, and flexibility, and adapting to the needs of different practical scenarios.
Smart Images

Figure CN119885176B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computing software technology and relates to a method for determining the source of open-source code for airborne software. Background Technology
[0002] With the increasing prevalence of computers, aviation equipment is becoming more and more reliant on computer systems with diverse functions. As we all know, programs are the core of computer systems, and the source code that generates those programs is crucial for their correct and stable operation. Today, open-source software is widely used, and the security of the large amount of open-source code incorporated into it is becoming increasingly important. Typically, security-related information for open-source software is released by its development and maintenance teams; therefore, knowing the origin of open-source code is of great significance for open-source security.
[0003] As open-source software becomes increasingly widely used, cross-referencing between open-source programs has become more frequent, making it increasingly difficult to pinpoint the source of open-source code. Currently, for a piece of open-source code, technical means can be used to search for it in open-source code repositories. However, identifying which parts of the search results are references and which are the true source requires professional expertise and manual judgment, which is difficult to accomplish when there are many search results or a large number of open-source code pieces whose sources need to be determined.
[0004] When a piece of open-source code is contained in multiple open-source projects, most existing methods use the creation time of the open-source projects, or the creation time of the file containing that piece of open-source code within those projects, as the basis for determining the source. That is, the earlier creation is considered the source of the open-source code, and the later creation is considered the referencing party.
[0005] Existing technologies can typically only search for a piece of open-source code in an open-source code repository, but there is a lack of effective methods to determine which search result is the true source of that open-source code.
[0006] Existing methods mostly rely on time, selecting the oldest project from the results as the source, but this approach is inaccurate. For example, some open-source projects, especially those with long time spans, may change their hosting platforms, such as migrating from SourceForge to GitHub. Although the project still exists on the old platform, its development and maintenance, especially the release of security information, are carried out on the new platform. From a security perspective, we should use projects on the new platform as the source, paying more attention to the activities of projects on the new platform, rather than focusing on projects on the older platform. Summary of the Invention
[0007] The purpose of this invention is to address the critical importance of understanding the source of open-source code for open-source security. This invention proposes a method for determining the source of open-source code in airborne software. The method comprehensively analyzes various metadata about open-source projects, including creation time, effectively avoiding the accuracy and reasonableness issues arising from existing methods that rely solely on time. Furthermore, the method allows for flexible adjustment of the analysis focus based on actual usage scenarios, making the analysis results more relevant to real-world situations. This method can more accurately locate the source project of open-source code, providing a fundamental guarantee for timely acquisition of open-source security information and effective open-source security governance.
[0008] Technical solution
[0009] A method for determining the source code of airborne software open source includes the following steps:
[0010] Step 1: Search open-source code repositories to obtain all sources of the input source code;
[0011] Step 2: Extract key information from each source in the search results of Step 1 and construct a keyword dictionary of key text information;
[0012] Step 3: Based on the key information extracted in Step 2 and the constructed keyword dictionary, score the source.
[0013] Step 4: Based on the scoring results of Step 3, select the source of the input source code.
[0014] Furthermore, in step 1, the open-source code repository is a storage that contains all open-source code that can be used as the retrieval object, and its coverage depends on the specific needs; the specific form of the open-source code repository is a database or file storage plus a text retrieval engine.
[0015] Step 1 specifically includes the following steps:
[0016] Step 1.1 Open source code feature extraction: First, feature extraction is performed on the source code in the code repository, mapping the code into feature vector representation;
[0017] Step 1.2 Calculate the similarity between different code repositories using the extracted code feature vectors;
[0018] Step 1.3 Based on the similarity calculation results, filter to obtain all possible open-source code sources for the input source code.
[0019] Furthermore, in step 1.1, the code is mapped to a feature vector representation using either the bag-of-words model or the n-gram model. The bag-of-words model represents each text as a fixed-length vector, where each dimension represents a word in the vocabulary, its frequency of occurrence in the text, or other statistical values. The n-gram model represents the text as a sequence of n adjacent words. The specific process of mapping the code to a feature vector representation using either the bag-of-words model or the n-gram model is as follows:
[0020] (1) Data preprocessing: Cleaning, word segmentation, and stop word removal of the original text to obtain a set of texts;
[0021] (2) Constructing a vocabulary: Create a vocabulary of all words that appear in the text set, and record the number of times each word appears in the text set or other statistical values. Other statistical values refer to the part of the source metadata that can identify the source features;
[0022] (3) Feature representation: Construct feature vectors based on the vocabulary and use bag-of-words model or n-gram model to represent the feature vectors;
[0023] (4) Feature selection: Use feature selection methods to select features that can effectively distinguish different categories;
[0024] (5) Feature scaling: Scaling the feature vectors to the same scale;
[0025] (6) Training the classifier: Use feature vectors to train the classifier.
[0026] Further, in step 1.2, the similarity between different code repositories is calculated using cosine similarity or Jaccard similarity calculation methods.
[0027] Furthermore, in step 2, the key information of the source refers to the part of the source's metadata that can identify the source characteristics, including but not limited to the open source project name, the name of the open source project developer or development team, the open source project creation time, the open source project popularity, and the file path of the input source code in the open source project.
[0028] Key text information refers to key information in the source that is textual, including but not limited to the name of the open source project, the name of the open source project developer or development team, and the file path of the input source code in the open source project.
[0029] A keyword dictionary refers to the information of keywords left after text processing of key information in a text. Its content includes keywords and their weights.
[0030] Furthermore, the steps for constructing a keyword dictionary include:
[0031] 2.1 Summary: Combine key information from all sources into one file;
[0032] 2.2 Text Processing: Format and filter the key text information summarized in 2.1, specifically including the following steps:
[0033] 2.2.1 Formatting: Standardize the format of key information by segmenting words, removing delimiters, and converting case.
[0034] 2.2.2 Filtering: Remove words from the key information of the text that cannot identify source features;
[0035] 2.3 Statistics: The words remaining after text processing are used as keywords, and the frequency of each keyword is counted;
[0036] 2.4 Weighting: Assign a weight to each keyword; the weighting algorithm follows the principles below and is designed according to specific needs. The weight is based on the frequency of occurrence; the more times a keyword appears, the greater its weight. The weight takes into account the importance of the key information in the text containing the keyword to determining the source of the input source code. The more important the key information in the text, the greater the weight of the keyword appearing in it.
[0037] Furthermore, step 3 specifically includes the following steps:
[0038] Step 3.1 Scoring the key information from the source includes the following steps:
[0039] Step 3.1.1 Score the key information in the source text;
[0040] Step 3.1.2: Score the non-textual key information from the source;
[0041] Step 3.2: Scoring the source.
[0042] Furthermore, step 3.1.1 scores the key information in the source text using the following method: a score S for a specific key piece of information in the source text. i Defined as
[0043] S i = ∑ k k∈K C k × W k (1)
[0044] Where K is the set of keywords in the key information, k is the keyword in the set, and C k W represents the number of times the keyword appears in the key information of the text. kThe weight of the keyword stored in the keyword dictionary is the weight of the keyword in the key information of the text. The score of the key information of the text is the weighted sum of the number of times each keyword appears in the key information. The weight is the weight of the corresponding keyword in the keyword dictionary.
[0045] In step 3.1.2, the scoring rules for non-textual key information follow the principle that the more it reflects the source, the higher the score.
[0046] Furthermore, step 3.2 scores the source in the following way:
[0047] Source rating S o Defined as:
[0048] S o =∑ i i∈I S i / Max(S i ) × W i (2)
[0049] Where I represents the set of key information from this source, i represents the key information in this set, and S... i For this key information, Max(S) is used to score it. i ) for each source of S i The maximum value, W i The weight of this key information is the weighted sum of the normalized scores of each key information in the source. The weight of key information varies depending on the requirements. The more important the key information is to the source of the input source code, the higher its weight. According to this calculation formula, the more complete the key information of the source, the higher its score.
[0050] Furthermore, step 4 is to use the source with the highest score as the source of the input source code.
[0051] Beneficial effects
[0052] This invention proposes a method for determining the source of open-source code. For a code file or its code snippets, the method attempts to search for the source code in an open-source code repository. By extracting key information from each source in the search results, the method evaluates each source based on this key information to determine which source is the true origin. Implementing this method enables automatic determination of the source of open-source code, eliminating manual intervention, improving efficiency, and contributing to open-source software security. Specifically, it offers the following advantages:
[0053] 1. The method described in this invention can effectively determine the true source of open-source code from multiple sources.
[0054] 2. By using the method described in this invention, the source of open-source code can be automatically determined, eliminating human intervention and improving the efficiency of the determination. Attached Figure Description
[0055] Figure 1 This is a flowchart of the method of the present invention;
[0056] Figure 2 This is a flowchart of the keyword dictionary generation process of the present invention;
[0057] Figure 3 This is a flowchart of the scoring process for the source of this invention. Detailed Implementation
[0058] The features and illustrative embodiments of various aspects of the present invention will now be described in detail. Numerous specific details are set forth in the following detailed description to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention may be practiced without requiring some of these specific details. The following description of embodiments is merely intended to provide a better understanding of the invention by illustrating examples of the invention. The invention is by no means limited to any specific setups and methods set forth below, but covers any improvements, substitutions, and modifications to structures, methods, and devices without departing from the spirit of the invention. Well-known structures and techniques are not shown in the drawings and the following description to avoid unnecessarily obscuring the invention.
[0059] It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other, and the various embodiments can be referenced and cited in each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0060] Example 1:
[0061] This invention proposes a method for determining the source code of open-source aircraft software, which mainly includes:
[0062] Step 1: Search open-source code repositories to obtain all sources of the input source code;
[0063] Step 2: Extract key information from each source in the search results of Step 1 and construct a keyword dictionary of key text information;
[0064] Step 3: Based on the key information extracted in Step 2 and the constructed keyword dictionary, score the source.
[0065] Step 4: Based on the scoring results of Step 3, select the source of the input source code.
[0066] The main process of the method of the present invention is as follows: Figure 1 As shown.
[0067] The open-source code repository described in this method is a storage containing all open-source code that can be used as the retrieval object. Its coverage depends on specific needs; for example, it could include all project code on the GitHub platform or all project code on the Gitee platform. The specific form of the open-source code repository can be various databases, or it can be a file storage system combined with a text retrieval engine.
[0068] Step 1: Search open-source code repositories to obtain all sources of the input source code, including:
[0069] 1. Open-source code feature extraction: First, feature extraction is performed on the source code in the code repository. Methods such as the bag-of-words model and n-gram model are used to map the code into feature vector representations. These feature vectors can help quantify the features of the code, laying the foundation for subsequent similarity calculations.
[0070] Among them, the bag-of-words model and the n-gram model are commonly used methods for text feature extraction. Their basic steps are as follows:
[0071] (1) Data preprocessing: Cleaning, word segmentation, and stop word removal are performed on the original text to obtain a set of texts.
[0072] (2) Constructing a vocabulary: Create a vocabulary of all words that appear in the text set, and record the number of times each word appears in the text set or other statistical values. Other statistical values refer to the part of the source metadata that can identify the source features;
[0073] (3) Feature representation: Construct feature vectors based on the vocabulary. The bag-of-words model or the n-gram model can be used to represent the feature vectors.
[0074] a. Bag-of-Words Model: Each text is represented as a fixed-length vector, where each dimension represents a word from the vocabulary, its frequency in the text, or other statistical values. For example, if the vocabulary contains 1000 words, then the feature vector for each text is a 1000-dimensional vector.
[0075] n-gram model: Representing text as a sequence of n adjacent words is called an n-gram. It typically uses bigram or trigram models (i.e., n=2 or n=3) to better capture local information in the text. For example, using a bigram model, the text "thecatsatonthemat" can be represented as a sequence of six bigrams: {thecat,catsat,saton,onthe,themat}. Then, feature vectors can be constructed by counting the frequency or other statistical values of each n-gram in the text.
[0076] (4) Feature selection: Since the vocabulary is usually very large, feature selection methods can be used to select some features that can effectively distinguish different categories. For example, for a vocabulary from the same open source community, ignore the community source feature; for a vocabulary from different open source communities, if the project time is the same, ignore the project time feature and consider features such as community popularity and resource richness.
[0077] (5) Feature scaling: Scaling the feature vectors to the same scale to avoid some features having too much influence on the bag-of-words model or n-gram model.
[0078] (6) Training the classifier: Train the classifier using feature vectors.
[0079] 2. Similarity Calculation: In this step, the extracted code feature vectors are used to calculate the similarity between different code repositories using methods such as cosine similarity and Jaccard similarity. This helps determine the degree of similarity between the code in two repositories, thus enabling further analysis of the relationship between them. Cosine similarity and Jaccard similarity are commonly used methods for calculating text similarity.
[0080] (1) Cosine similarity is a method for calculating similarity based on the vector space model. It measures the degree of similarity between two vectors by calculating the cosine of the angle between them. The specific calculation method is as follows: Suppose there are two vectors A and B, which have the same dimension, n. Then their cosine similarity can be expressed as: cos(A,B)=(A·B) / (||A||*||B||), where A·B represents the dot product of the two vectors, and ||A|| and ||B|| represent the magnitudes (also called the norms) of vectors A and B. The more similar the two vectors are, the smaller their angle is, and the closer the cosine value is to 1; conversely, the less similar the two vectors are, the larger their angle is, and the closer the cosine value is to 0.
[0081] (2) Jaccard similarity is a method for calculating the similarity of sets, typically used to compare the degree of overlap between elements in two sets. The specific calculation method is as follows: Assuming there are two sets A and B, containing m and n elements respectively, their Jaccard similarity can be expressed as: J(A,B)=|A∩B| / |A∪B| where A∩B represents the intersection of sets A and B, and |A∩B| represents the number of elements in the intersection; A∪B represents the union of sets A and B, and |A∪B| represents the number of elements in the union. The more similar the two sets are, the larger the proportion of elements in their intersection, and the closer the Jaccard similarity is to 1; conversely, the less similar the two sets are, the smaller the proportion of elements in their intersection, and the closer the Jaccard similarity is to 0.
[0082] 3. Filter based on similarity to obtain all possible open source code sources for the input source code.
[0083] Step 2: Extract key information from each source in the search results of Step 1 and construct a keyword dictionary of key text information;
[0084] Key information about the source refers to the part of the source's metadata that can identify the source's characteristics, including but not limited to the open source project name, the name of the open source project developer or development team, the open source project creation time, the open source project's popularity, and the file path of the input source code in the open source project.
[0085] Key text information refers to the key information in the source that is textual, such as the name of the open source project, the name of the open source project developer or development team, and the file path of the input source code in the open source project.
[0086] A keyword dictionary refers to the information of keywords remaining after text processing of key information in a text. Its content includes keywords and their weights. See step 2 below for details on constructing a keyword dictionary.
[0087] The steps for constructing the keyword dictionary include:
[0088] 1. Summarize. Combine key text information from all sources.
[0089] 2. Text Processing. Format and filter the key text information summarized in step 1.
[0090] 2.1 Formatting. Use technical means to standardize the format of key information. These means include word segmentation, delimiter removal, and case conversion.
[0091] 2.2 Filtering. Remove words from the key information of the text that cannot identify source characteristics. Such as meaningless function words, words that frequently appear in key information but lack characteristics (such as "src" in the file path of the input source code in an open source project, etc.).
[0092] 3. Statistics. The words remaining after text processing are used as keywords. The frequency of each keyword is then counted.
[0093] 4. Weighting. Assign a weight to each keyword. The weighting algorithm can be designed according to the following principles and specific needs: Weights are based on frequency of occurrence; the more frequently a keyword appears, the greater its weight. Weights also consider the importance of the key information in the text containing the keyword to determining the source of the input source code; the more important the key information, the greater the weight of the keyword appearing within it. For example, if, based on specific needs, the name of the open-source project is more important than the file path of the input source code within the open-source project, then keywords appearing in the open-source project name will have a higher weight, while other keywords appearing in the file path of the input source code within the open-source project will have a lower weight.
[0094] The main process of generating a keyword dictionary is as follows: Figure 2 As shown.
[0095] Step 3: The steps for scoring the source as described in the method include:
[0096] 1. Scoring of key information from the source.
[0097] 1.1 Scoring of key information in the source text.
[0098] The score S of a key piece of information in a source text i Defined as
[0099] S i = ∑ k k∈K C k × W k (Formula 1)
[0100] Where K is the set of keywords in the key information, k is the keyword in the set, and C k W represents the number of times the keyword appears in the key information of the text. k This represents the weight of the keyword stored in the keyword dictionary. In other words, the score of the key information in the text is the weighted sum of the frequency of each keyword appearing in the key information, with the weight being the weight of the corresponding keyword in the keyword dictionary.
[0101] 1.2 Scoring of non-textual key information from the source.
[0102] The scoring rules for non-textual key information vary depending on the meaning of the key information. When designing the scoring rules, the principle is that the more the information reflects its origin, the higher the score. For example, the earlier an open-source project was created, the higher its score; the more popular an open-source project is, the higher its score.
[0103] 2. Rating the source.
[0104] Source rating S o Defined as
[0105] S o =∑ i i∈I S i / Max(S i ) × W i (Formula 2)
[0106] Where I represents the set of key information from this source, i represents the key information in this set, and S... i For this key information, Max(S) is used to score it. i ) for each source of S i The maximum value, W i This refers to the weight of the key information. That is, the source score is the weighted sum of the normalized scores of each key piece of information from the source. The weight of key information varies depending on the requirements; the more important the key information is to determining the source of the input source code, the higher its weight. According to this calculation formula, the more complete the key information of the source, the higher its score.
[0107] The main process for scoring sources is as follows: Figure 3 As shown.
[0108] Step 4: Finally, the source with the highest score will be used as the source of the input source code.
[0109] Example 2:
[0110] This embodiment demonstrates how to use the method described in this invention to determine the source of a piece of source code.
[0111] To ensure that the search results for the input source code in the open-source code repository are not empty, this embodiment uses a piece of open-source code, specifically the file libavcodec / mpeg4video.h from version v0.6.1 of the open-source project FFmpeg, as the input source code. The steps to determine its source are as follows:
[0112] 1. Build an open-source codebase.
[0113] 1.1 This embodiment builds an open-source code repository based on open-source projects on the GitHub open-source code platform. The code of the open-source projects is downloaded from GitHub to the local disk using the `git` command, and metadata about each open-source project is crawled using a web crawler.
[0114] 1.2 In this embodiment, a MySQL database is used to store the open-source code repository, and the code is stored at the file level. A project information table (`project`) is created to store the metadata of the open-source projects crawled in 1.1. A file information table (`file`) is created to store the path, MD5 hash, and the corresponding open-source project of the code files downloaded in 1.1.
[0115] 2. Search open-source code repositories to obtain all sources of the input source code.
[0116] 2.1 Convert the input source code to MD5. In this embodiment, the input source code is the file mpeg4video.h, and its MD5 is "4e94ceb168e1592ffd8c132cab2128fc".
[0117] 2.2. By querying the `file` table using the MD5 hash and simultaneously associating it with the `project` table, metadata about all sources of the input source code is obtained. In this embodiment, this metadata includes: open-source project name, open-source project author, number of stars for the open-source project, open-source project license type, and the file path of the input source code within the open-source project. The results of the above query in the open-source code repository constructed in this embodiment are shown in Table 1.
[0118]
[0119]
[0120] Table 1 Source List
[0121] 3. Extract key information from each source and construct a keyword dictionary of key information in the text.
[0122] 3.1 In this embodiment, the following information from the source metadata is selected as the key information of the source: open source project name, open source project author, open source project popularity, and the file path of the input source code in the open source project. Among them, the open source project name, open source project author, and the file path of the input source code in the open source project are text key information, while the open source project popularity is non-text key information.
[0123] 3.2 Construct a keyword dictionary for key information in the text.
[0124] 3.2.1 Summary. Combine the three columns of data in Table 1: open source project name, open source project author, and file path of the input source code in the open source project, to form a text set Ti[xbmc-antiquated,FFMpeg,atv2,xbmc,lib / ffmpeg / libavcodec / mpeg4video.h,...].
[0125] 3.2.2 Formatting. The text in Ti is split into basic words or numbers using delimiters such as "-", " / ", and ".". The delimiters are removed, and all letters in the text are converted to lowercase, resulting in a set of text Tf[xbmc,antiquated,ffmpeg,atv2,xbmc,lib,ffmpeg,libavcodec,mpeg4video,...].
[0126] 3.2.3 Filtering. Remove meaningless words [0, 6, x, h, ...] from the Tf file. Further remove generic words that frequently appear in key information and lack distinctive features. In the open-source codebase built in this embodiment, these words are [lib, src, release, modules, ...]. After filtering, a set of keyword text T[xbmc, antiquated, ffmpeg, atv2, xbmc, ffmpeg, libavcodec, mpeg4video, ...] is obtained.
[0127] 3.2.4. Statistics. Count the number of times each keyword appears in T to obtain the keyword list K [(ffmpeg:14),(xbmc:8),(tomasen:1),...].
[0128] 3.2.5 Weighting. In this embodiment, the weight W of keyword k is... k Defined as
[0129] W k =C k ×P
[0130] Among them, C k P represents the number of times the keyword appears, and P is an adjustment coefficient. In this embodiment, the open-source project name is used as an important key information for determining the source of the input source code. Therefore, if the keyword appears in the open-source project name of each source, then P = 1.2, otherwise P = 1. The final keyword dictionary D[(ffmpeg:16.8),(xbmc:9.6),(tomasen:1),...] is obtained.
[0131] 4. Rate the source.
[0132] 4.1 Scoring of key information from the source. The results are shown in Table 2.
[0133] 4.1.1 Scoring of key text information from sources. Calculate the score of key text information from each source according to formula (1).
[0134] 4.1.2 Scoring of non-textual key information from the source. In this embodiment, the non-textual key information from the source is the number of stars in the open-source project, and its score is defined as the number of stars itself.
[0135]
[0136]
[0137] Table 2. Scoring of Key Information in the Text
[0138] 4.2. Source Scoring. Calculate the score for each source according to Formula 2. In this embodiment, the open-source project name is considered more important than other key information for determining the source of the input source code. Therefore, for the open-source project name, W... i =1.2, W for other key information i =1. The scoring results from each source are shown in Table 3.
[0139] id Open source project name Open source project source rating 1 xbmc-antiquated 2.390 2 FFMpeg 2.031 3 atv2 1.705 4 FFmpeg 3.662 5 EasyTomato 0.931 6 xbmc-rbp 2.129 7 plex-linux 1.279 8 asus-rt-n66u-merlin 1.187 9 dsl-n55u-bender 1.102 10 DreamShell 1.322 11 splayer 1.007 12 rt-n56u 1.132
[0140] Table 3 Source Rating
[0141] 5. Select the source of the input source code. Based on the scoring results obtained in 4.2, the method determines that FFmpeg is the source of the input source code. This result is consistent with the actual result.
[0142] If traditional time-based methods for determining open-source origins are used, the earliest project listed above would be xbmc-antiquated, which is inconsistent with reality. Compared to traditional methods, this method provides a more accurate determination of the open-source origin.
[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should be covered within the protection scope of the present invention.
Claims
1. A method for determining the source of open-source code for airborne software, characterized in that, Includes the following steps: Step 1: Search open-source code repositories to obtain all sources of the input source code. In Step 1, an open-source code repository is a storage containing all open-source code that can be used as the search target; its coverage depends on specific needs. The specific form of an open-source code repository is a database or file storage combined with a text search engine, including the following steps: Step 1.1 Open source code feature extraction: First, feature extraction is performed on the source code in the code repository, mapping the code into feature vector representation; Step 1.2 Calculate the similarity between different code repositories using the extracted code feature vectors; Step 1.3 Based on the similarity calculation results, filter to obtain all possible open-source code sources for the input source code; Step 2: Extract key information from the search results in Step 1 and construct a keyword dictionary of text key information. In Step 2, the key information of the source refers to the part of the source's meta-information that can identify the source characteristics, including but not limited to the open source project name, the name of the open source project developer or development team, the open source project creation time, the popularity of the open source project, and the file path of the input source code in the open source project. Key text information refers to key information in the source that is textual, including but not limited to the name of the open source project, the name of the open source project developer or development team, and the file path of the input source code in the open source project. A keyword dictionary refers to the information of keywords left after text processing of key information in a text. Its content includes keywords and their weights. Step 3: Based on the key information extracted in Step 2 and the constructed keyword dictionary, score the source, including the following steps: Step 3.1 Scoring the key information from the source includes the following steps: Step 3.1.1 Scoring the key information in the source text; Step 3.1.1 scoring the key information in the source text is conducted in the following way: The score S of a key piece of information in a source text i Defined as S i = ∑ k k∈K C k × W k (1) Where K is the set of keywords in the key information, k is the keyword in the set, and C k W represents the number of times the keyword appears in the key information of the text. k The weight of the keyword stored in the keyword dictionary is the weight of the keyword in the key information of the text. The score of the key information of the text is the weighted sum of the number of times each keyword appears in the key information. The weight is the weight of the corresponding keyword in the keyword dictionary. Step 3.1.2 Scoring of non-textual key information from the source; In Step 3.1.2, the scoring rules for non-textual key information follow the principle that the more it reflects the source, the higher the score; Step 3.2 Scoring of sources: Sources are scored using the following methods: Source rating S o Defined as: S o =∑ i i∈I S i / Max(S i ) × W i (2) Where I represents the set of key information from this source, i represents the key information in this set, and S... i For this key information, Max(S) is used to score it. i ) for each source of S i The maximum value, W i The weight of this key information is the weighted sum of the normalized scores of each key information in the source. The weight of key information varies depending on the requirements. The more important the key information is to the source of the input source code, the higher its weight. According to this calculation formula, the more complete the key information of the source, the higher its score. Step 4: Based on the scoring results of Step 3, select the source of the input source code.
2. The method for determining the source of open-source aircraft software according to claim 1, characterized in that, In step 1.1, the code is mapped to a feature vector representation using either the bag-of-words model or the n-gram model. The bag-of-words model represents each text as a fixed-length vector, where each dimension represents a word in the vocabulary, its frequency of occurrence, or other statistical value within the text. The n-gram model represents the text as a sequence of n adjacent words. The specific process of mapping the code to a feature vector representation using either the bag-of-words model or the n-gram model is as follows: (1) Data preprocessing: Cleaning, word segmentation, and stop word removal of the original text to obtain a set of texts; (2) Constructing a vocabulary: Create a vocabulary of all words that appear in the text set, and record the number of times each word appears in the text set or other statistical values. Other statistical values refer to the part of the source metadata that can identify the source features; (3) Feature representation: Construct feature vectors based on the vocabulary and use bag-of-words model or n-gram model to represent the feature vectors; (4) Feature selection: Use feature selection methods to select features that can effectively distinguish different categories; (5) Feature scaling: Scaling the feature vectors to the same scale; (6) Training the classifier: Use feature vectors to train the classifier.
3. The method for determining the source of open-source aircraft software according to claim 2, characterized in that, Step 1.2: Calculate the similarity between different code repositories using cosine similarity or Jaccard similarity calculation methods.
4. The method for determining the source of open-source aircraft software according to claim 3, characterized in that, The steps to build a keyword dictionary include: 2.1 Summary: Combine key information from all sources into one file; 2.2 Text Processing: Format and filter the key text information summarized in 2.1, specifically including the following steps: 2.2.1 Formatting: Standardize the format of key information by segmenting words, removing delimiters, and converting case. 2.2.2 Filtering: Remove words from the key information of the text that cannot identify source features; 2.3 Statistics: The words remaining after text processing are used as keywords, and the frequency of each keyword is counted; 2.4 Weighting: Assign a weight to each keyword; the weighting algorithm follows the principles below and is designed according to specific needs. The weight is based on the frequency of occurrence; the more times a keyword appears, the greater its weight. The weight takes into account the importance of the key information in the text containing the keyword to determining the source of the input source code. The more important the key information in the text, the greater the weight of the keyword appearing in it.
5. The method for determining the source of open-source aircraft software according to claim 4, characterized in that, Step 4 is to use the highest-rated source as the source of the input source code.
Citation Information
Patent Citations
Document keyword extraction method based on hypergraph random walk
CN114065758A
Text classification method based on keyword extraction and attention mechanism
CN115168580A