Method for judging source of open source code of aviation airborne software
By building a keyword dictionary and scoring mechanism, comprehensively analyzing the meta information of open source projects, the problem of inaccurate judgment of open source code in the existing technology is solved, and automated and efficient judgment of open source code source code is achieved.
Patent Information
- Application Number
- CN202411957367.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-27
- Publication Date
- 2025-04-25
- Estimated Expiration
- 2044-12-27
AI Technical Summary
It is difficult for the prior art to accurately determine the source of open source code, especially when open source software frequently references to each other, traditional methods rely on creation time and lead to inaccurate judgments.
By building a keyword dictionary and scoring mechanism, comprehensively analyze the meta information of open source projects, including project name, developer, popularity, etc., cosine similarity and Jaccard similarity calculate the code warehouse similarity, and automatically determine the source of open source code.
Automatic judgment of open source code source is realized, judgment efficiency is improved, manual intervention is reduced, and judgment accuracy is improved.
Smart Images

Figure CN119885176A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of computing software and relates to a method for determining the source of open source code of aviation airborne software. Background Art
[0002] As the application of computers becomes more and more popular, aviation equipment is increasingly dependent on computer systems with various functions. As we all know, programs are the core part of computer systems, and the source code that generates programs is the key to the correct and stable operation of programs. Nowadays, open source software is widely used, and the security of a large number of open source codes introduced in the source code is becoming more and more important. Usually, the security-related information of open source software is released by its development and maintenance team, so knowing the source of the open source code is of great significance to open source security.
[0003] As open source software is increasingly used, mutual references between open source software become more frequent, and it is increasingly difficult to confirm the source of open source code. At present, for a piece of open source code, it is possible to search for the code from the open source code library through technical means. However, which of the search results are the references and which are the real sources need to be manually determined through professional knowledge, which is difficult to accomplish when there are many search results or a large number of open source codes that need to determine the source.
[0004] When a piece of open source code is included in multiple open source projects, most existing methods use the creation time of the open source project or the creation time of the file containing the open source code in the open source project as the basis for determining the source. That is, the one created first is the source of the open source code, and the one created later is the reference.
[0005] Existing technical means can usually only search for a certain section of open source code from the open source code library, but there is a lack of effective determination method as to which search result is the real source of the open source code.
[0006] Most existing methods are based on time, selecting the earliest created project in the results as the source, but this method is not accurate. For example, some open source projects, especially those with a long time span, will change the project hosting platform, such as migrating from SourceForge to GitHub. Although the projects on the old platform still exist, the development and maintenance of the projects, especially the release of security information, are all carried out on the new platform. From a security-related perspective, we should take the projects on the new platform as the source and pay more attention to the trends of the projects on the new platform, instead of focusing on the projects on the old platform that were created earlier. Summary of the invention
[0007] Purpose of the present invention: Knowing the source of open source code is of great significance to open source security. The present invention proposes a method for determining the source of open source code for aviation airborne software. The method comprehensively analyzes various open source project metadata including creation time, effectively avoiding the accuracy and rationality problems caused by existing methods that only consider time. At the same time, the method supports flexible adjustment of the focus of the analysis according to the actual usage scenario, so that the analysis results are more in line with the actual scenario. This method can more accurately locate the source project of the open source code, and provide a basic guarantee for timely acquisition of open source security information and effective open source security governance.
[0008] Technical Solution
[0009] A method for determining the source of open source code of aviation airborne software comprises the following steps:
[0010] Step 1: Search the open source code repository to obtain all sources of input source code;
[0011] Step 2: Extract key information from each source from the search results of step 1, and construct a keyword dictionary of the key information of the text;
[0012] Step 3: Based on the key information extracted in step 2 and the keyword dictionary constructed, score the source;
[0013] Step 4: Based on the scoring result of step 3, select the source of the input source code.
[0014] Further, in step 1, the open source code base is a storage containing all open source codes that are search objects, and its coverage depends on specific needs; the specific form of the open source code base is a database or file storage plus a text search engine;
[0015] Step 1 specifically includes the following steps:
[0016] Step 1.1 Open source code feature extraction: First, extract features from the source code in the code repository and map the code into a feature vector representation;
[0017] Step 1.2 uses the extracted code feature vectors to calculate the similarity between different code repositories;
[0018] Step 1.3: Filter all possible open source sources of the input source code based on the similarity calculation results.
[0019] Furthermore, in step 1.1, the code is mapped to a feature vector representation using a bag-of-words model or an n-gram model, wherein the bag-of-words model represents each text as a vector of fixed length, and each dimension in the vector represents a word in the vocabulary, the number of occurrences in the text, or other statistical values; the n-gram model represents the text as a sequence of n adjacent words. The specific process of mapping the code to a feature vector representation using the bag-of-words model or the n-gram model is as follows:
[0020] (1) Data preprocessing: Clean, segment, and remove stop words from the original text to obtain a set of texts;
[0021] (2) Constructing a vocabulary: Construct a vocabulary of all words that appear in the text set, recording the number of times each word appears in the text set or other statistical values. Other statistical values refer to the part of the source meta-information that can identify the source characteristics;
[0022] (3) Feature representation: construct feature vectors based on the vocabulary and use the bag-of-words model or n-gram model to represent the feature vectors;
[0023] (4) Feature selection: Use feature selection methods to select features that can effectively distinguish different categories;
[0024] (5) Feature scaling: scaling feature vectors to the same scale;
[0025] (6) Train the classifier: Use the feature vector to train the classifier.
[0026] Furthermore, in step 1.2, the similarity between different code repositories is calculated using a cosine similarity or Jaccard similarity calculation method.
[0027] Furthermore, in step 2, the key information of the source refers to the part of the source's meta-information that can identify the source's characteristics, including but not limited to the name of the open source project, the name of the open source project developer or development team, the creation time of the open source project, the popularity of the open source project, and the file path of the input source code in the open source project;
[0028] Text key information refers to the key information of the source, the content of which is the key information of the text, including but not limited to the name of the open source project, the name of the open source project developer or development team, and the file path of the input source code in the open source project;
[0029] The keyword dictionary refers to the keyword information left after text processing of the key information of the text, and its content includes keywords and their weights.
[0030] Further, the steps of constructing a keyword dictionary include:
[0031] 2.1 Summarize: combine the key information of the text from all sources;
[0032] 2.2 Text processing: Format and filter the key information of the text summarized in 2.1, including the following steps:
[0033] 2.2.1 Formatting: Unify the format of key information by cutting words, removing separators, and converting uppercase and lowercase characters;
[0034] 2.2.2 Filtering: Remove words that cannot identify source characteristics from the key information of the text;
[0035] 2.3 Statistics: The words left after text processing are used as keywords, and the number of times each keyword appears is counted;
[0036] 2.4 Weighting: Assign a weight to each keyword; the weight algorithm is based on the following principles and is designed according to specific needs. The weight is based on the number of occurrences. The more times a keyword appears, the greater its weight. The weight takes into account the importance of the key information in the text where the keyword is located in determining the source of the input source code. The more important the key information in the text, the greater the weight of the keyword appearing in it.
[0037] Further, step 3 specifically includes the following steps:
[0038] Step 3.1: Score the key information of the source, including the following steps:
[0039] Step 3.1.1 Score the key information of the source text;
[0040] Step 3.1.2: Score the non-text key information of the source;
[0041] Step 3.2: Score the sources.
[0042] Furthermore, step 3.1.1 scores the key information of the source text in the following way: the score S of a key information of the source text i Defined as
[0043] S i = ∑ k k∈K C k × W k (1)
[0044] Among them, K is the keyword set in the key information, k is the keyword in the set, and C k is the number of times the keyword appears in the key information of the text, W kis the weight of the keyword stored in the keyword dictionary, that is, the score of the key information of the text is the weighted sum of the number of occurrences of each keyword in the key information, and the weight is the weight of the corresponding keyword in the keyword dictionary;
[0045] In step 3.1.2, the scoring rule for non-text key information follows the principle that the more it reflects the source, the higher the score.
[0046] Further, step 3.2 scores the sources in the following way:
[0047] Source rating S o Defined as:
[0048] S o =∑ i i∈I S i / Max(S i ) × W i (2)
[0049] Where I is the key information set of the source, i is the key information in the set, S i is the score of the key information, Max(S i ) is the S of each source i The maximum value, W i is the weight of the key information, that is, the score of the source is the weighted sum of the normalized scores of the key information of the source; the weight of the key information varies according to different requirements. The more important the key information is for determining the source of the input source code, the higher its weight. According to the calculation formula, the more complete the key information of the source is, the higher its score is.
[0050] Further, step 4 is to use the source with the highest score as the source of the input source code.
[0051] Beneficial Effects
[0052] The present invention proposes a method for determining the source of open source code. For a code file or a code fragment therein, the method attempts to search for the source code from an open source code library, extracts key information of each source in the search results, evaluates each source based on the key information, and determines which source is the real source. By implementing this method, the source of open source code can be automatically determined, manual participation can be removed, the efficiency of determination can be improved, and the open source security of software can be promoted. Specifically, the advantages include the following:
[0053] 1. The method of the present invention can effectively determine the true source of open source code from multiple sources.
[0054] 2. By using the method of the present invention, the source of open source code can be automatically determined, human involvement can be eliminated, and the efficiency of determination can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 is a flow chart of the method of the present invention;
[0056] Figure 2 It is a flow chart of generating a keyword dictionary according to the present invention;
[0057] Figure 3 It is a source scoring flow chart of the present invention. DETAILED DESCRIPTION
[0058] The features and illustrative embodiments of various aspects of the present invention will be described in detail below. In the detailed description below, many specific details are proposed to provide a comprehensive understanding of the present invention. However, it is obvious to those skilled in the art that the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention. The present invention is by no means limited to any specific settings and methods proposed below, but covers any improvements, replacements and modifications of structures, methods, devices without departing from the spirit of the present invention. In the accompanying drawings and the following description, known structures and technologies are not shown to avoid unnecessary ambiguity in the present invention.
[0059] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other, and the embodiments can refer to and quote each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0060] Embodiment 1:
[0061] The present invention proposes a method for determining the source of open source code of aviation airborne software, which mainly includes:
[0062] Step 1: Search the open source code repository to obtain all sources of input source code;
[0063] Step 2: Extract key information from each source from the search results of step 1, and construct a keyword dictionary of the key information of the text;
[0064] Step 3: Based on the key information extracted in step 2 and the keyword dictionary constructed, score the source;
[0065] Step 4: Based on the scoring result of step 3, select the source of the input source code.
[0066] The main process of the method of the present invention is as follows Figure 1 shown.
[0067] The open source code library described in the method is a storage containing all open source codes that are searched, and its coverage depends on specific needs. For example, it can contain all project codes on the GitHub platform, or all project codes on the Gitee platform. The specific form of the open source code library can be various databases, or it can be a file storage plus a text search engine.
[0068] Step 1 Search the open source code repository to obtain all sources of input source code including:
[0069] 1. Open source code feature extraction: First, extract features from the source code in the code repository and map the code to feature vector representation using methods such as bag-of-words model and n-gram model. These feature vectors can help quantify the features of the code and lay the foundation for subsequent similarity calculations.
[0070] Among them, the bag-of-words model and the n-gram model are commonly used methods for text feature extraction. Their basic steps are as follows:
[0071] (1) Data preprocessing: Clean the original text, segment it, remove stop words, and obtain a set of text collections
[0072] (2) Constructing a vocabulary: Construct a vocabulary of all words that appear in the text set, recording the number of times each word appears in the text set or other statistical values. Other statistical values refer to the part of the source meta-information that can identify the source characteristics;
[0073] (3) Feature representation: Construct feature vectors based on the vocabulary. The bag-of-words model or n-gram model can be used to represent feature vectors.
[0074] a. Bag of Words Model: Each text is represented as a vector of fixed length, where each dimension in the vector represents a word in the vocabulary, the number of occurrences in the text, or other statistical values. For example, if the vocabulary contains 1,000 words, the feature vector of each text is a 1,000-dimensional vector.
[0075] bn-gram model: The text is represented as a sequence of n adjacent words, called n-gram. Usually a bigram or trigram model (i.e. n=2 or n=3) is used, which can better capture the local information in the text. For example, if a bigram model is used, the text "thecatsatonthemat" is represented as a sequence of six bigrams: {thecat, catsat, saton, onthe, themat}. Then, the feature vector can be constructed based on the number of occurrences or other statistical values of the n-grams that appear in each text.
[0076] (4) Feature selection: Since the vocabulary is usually very large, feature selection methods can be used to select some features that can effectively distinguish different categories. For example, for vocabulary from the same open source community, the community source feature is ignored; for vocabulary from different open source communities, if the project time is the same, the project time feature is ignored, and features such as community popularity and resource richness need to be considered.
[0077] (5) Feature scaling: Scale feature vectors to the same scale to avoid certain features having too much influence on the bag-of-words model or n-gram model.
[0078] (6) Training the classifier: Using feature vectors to train the classifier
[0079] 2. Similarity calculation: In this step, the extracted code feature vectors are used to calculate the similarity between different code repositories using methods such as cosine similarity and Jaccard similarity. This helps determine the degree of similarity between the codes of two code repositories, and further analyze the relationship between them. Among them, cosine similarity and Jaccard similarity are both commonly used methods for calculating text similarity.
[0080] (1) Cosine Similarity is a method for calculating similarity based on the vector space model. It measures the similarity between two vectors by calculating the cosine value of the angle between them. The specific calculation method is as follows: Suppose there are two vectors A and B, and their dimensions are the same, n dimensions respectively. Then their cosine similarity can be expressed as: cos(A,B) = (A·B) / (||A||*||B||), where A·B represents the dot product of the two vectors, and ||A|| and ||B|| represent the modulus length of vectors A and B (also called the norm of the vector). If two vectors are more similar, their angle is smaller and the cosine value is closer to 1; conversely, if two vectors are less similar, their angle is larger and the cosine value is closer to 0.
[0081] (2) Jaccard Similarity is a method for calculating set similarity, which is usually used to compare the degree of overlap of elements in two sets. The specific calculation method is as follows: Assuming there are two sets A and B, containing m and n elements respectively, their Jaccard similarity can be expressed as: J(A,B) = |A∩B| / |A∪B|, where A∩B represents the intersection of sets A and B, |A∩B| represents the number of elements in the intersection; A∪B represents the union of sets A and B, |A∪B| represents the number of elements in the union. If the two sets are more similar, the proportion of elements in their intersection will be greater, and the Jaccard similarity will be closer to 1; conversely, if the two sets are less similar, the proportion of elements in their intersection will be smaller, and the Jaccard similarity will be closer to 0.
[0082] 3. Filter all possible open source sources of the input source code based on similarity.
[0083] Step 2: Extract key information from each source from the search results of step 1, and construct a keyword dictionary of the key information of the text;
[0084] The key information of the source refers to the part of the source's meta-information that can identify the source's characteristics, including but not limited to the name of the open source project, the name of the open source project developer or development team, the creation time of the open source project, the popularity of the open source project, the file path of the input source code in the open source project, etc.
[0085] Text key information refers to the key information of the source, and the content is the key information of the text, such as the name of the open source project, the name of the open source project developer or development team, the file path of the input source code in the open source project, etc.
[0086] The keyword dictionary refers to the keyword information left after text processing of the key information of the text, and its content includes keywords and their weights. For details, see step 2 of building a keyword dictionary below.
[0087] The steps of constructing a keyword dictionary include:
[0088] 1. Summarize. Combine the key information from all sources.
[0089] 2. Text processing: Format and filter the key information of the text summarized in 1.
[0090] 2.1. Formatting. Use technical means to unify the format of key information. Technical means include word segmentation, delimiter removal, and case conversion.
[0091] 2.2. Filtering. Remove words that cannot identify source features in the key information of the text, such as meaningless function words, and words without features that often appear in key information (such as "src" in the file path of the input source code in an open source project).
[0092] 3. Statistics. The words left after text processing are used as keywords. The number of times each keyword appears is counted.
[0093] 4. Weighting. Assign a weight to each keyword. The weighting algorithm can be designed according to the following principles and specific needs. The weight is based on the number of occurrences. The more times it appears, the greater the weight. The weight takes into account the importance of the key information in the text where the keyword is located for determining the source of the input source code. The more important the key information in the text, the greater the weight of the keyword appearing in it. For example, if according to specific needs, the importance of the open source project name is greater than the importance of the file path of the input source code in the open source project, then the keyword appearing in the open source project name has a larger weight, and other keywords appearing in the file path of the input source code in the open source project have a smaller weight.
[0094] The main process of generating a keyword dictionary is as follows: Figure 2 shown.
[0095] Step 3: The steps for scoring sources as described in the methodology include:
[0096] 1. Score the key information of the source.
[0097] 1.1. Score the key information of the source text.
[0098] The score S of a key information of a text in the source i Defined as
[0099] S i = ∑ k k∈K C k × W k (Formula 1)
[0100] Among them, K is the keyword set in the key information, k is the keyword in the set, and C k is the number of times the keyword appears in the key information of the text, W k is the weight of the keyword stored in the keyword dictionary. That is, the score of the text key information is the weighted sum of the number of occurrences of each keyword in the key information, and the weight is the weight of the corresponding keyword in the keyword dictionary.
[0101] 1.2. Score the non-text key information of the source.
[0102] The scoring rules for non-text key information vary depending on the meaning of the key information. When designing the scoring rules, we follow the principle that the more the source is reflected, the higher the score. For example, the earlier the open source project was created, the higher the score; the more popular the open source project is, the higher the score is.
[0103] 2. Rate the sources.
[0104] Source rating S o Defined as
[0105] S o =∑ i i∈I S i / Max(S i ) × W i (Formula 2)
[0106] Where I is the key information set of the source, i is the key information in the set, S i is the score of the key information, Max(S i ) is the S of each source i The maximum value, W i is the weight of the key information. That is, the score of the source is the weighted sum of the normalized scores of each key information of the source. The weight of key information varies according to different requirements. The more important the key information is for determining the source of the input source code, the higher its weight. According to the calculation formula, the more complete the key information of the source is, the higher its score is.
[0107] The main process of scoring sources is as follows: Figure 3 shown.
[0108] Step 4: Finally, the source with the highest score is used as the source of the input source code.
[0109] Embodiment 2:
[0110] This embodiment demonstrates how to use the method of the present invention to determine the origin of a piece of source code.
[0111] In order to ensure that the search results of the input source code in the open source code library are not empty, this embodiment uses a piece of open source code, that is, the file libavcodec / mpeg4video.h in the v0.6.1 version of the open source project FFmpeg, as the input source code. The steps for determining its source are as follows:
[0112] 1. Build an open source code base.
[0113] 1.1. This embodiment builds an open source code base based on the open source projects in the open source code platform GitHub. The code of the open source project is downloaded from GitHub to the local disk through the git command, and the meta information of each open source project is crawled through the web crawler.
[0114] 1.2. This embodiment uses MySQL database as the storage of the open source code base, and the code storage granularity is file. Create a project information table project, and store the open source project metadata crawled in 1.1 in the table. Create a file information table file, and store the path, MD5, and open source project of the code file of the open source project downloaded in 1.1 in the table.
[0115] 2. Search open source code repositories to obtain all sources of input source code.
[0116] 2.1. Convert the input source code to MD5. In this embodiment, the input source code is the file mpeg4video.h, and its MD5 is "4e94ceb168e1592ffd8c132cab2128fc".
[0117] 2.2. Query the file table through the above MD5, and associate the project table at the same time to obtain the meta information of all sources of the input source code. In this embodiment, this meta information includes: the name of the open source project, the author of the open source project, the number of stars of the open source project, the license type of the open source project, and the file path of the input source code in the open source project. In the open source code library constructed in this embodiment, the results of the above query are shown in Table 1.
[0118]
[0119]
[0120] Table 1 Source list
[0121] 3. Extract key information from each source and build a keyword dictionary of key information in the text.
[0122] 3.1. In this embodiment, the following information in the source meta-information is selected as the key information of the source: the name of the open source project, the author of the open source project, the popularity of the open source project, and the file path of the input source code in the open source project. Among them, the name of the open source project, the author of the open source project, and the file path of the input source code in the open source project are text key information, and the popularity of the open source project is non-text key information.
[0123] 3.2. Build a keyword dictionary of key information in the text.
[0124] 3.2.1. Summary. The three columns of data in Table 1, namely, the name of the open source project, the author of the open source project, and the file path of the input source code in the open source project, are merged to form a set of text Ti[xbmc-antiquated,FFMpeg,atv2,xbmc,lib / ffmpeg / libavcodec / mpeg4video.h,...].
[0125] 3.2.2 Formatting. Cut the text in Ti into basic words or numbers with separators such as "-", " / ", ".", remove the separators, and convert all letters in the text to lowercase to obtain a set of text Tf[xbmc,antiquated,ffmpeg,atv2,xbmc,lib,ffmpeg,libavcodec,mpeg4video,...].
[0126] 3.2.3, Filtering. Remove meaningless words [0,6,x,h,...] in Tf, and further remove common words without characteristics that often appear in key information. In the open source code library constructed in this embodiment, these words are [lib,src,release,modules,...]. After filtering, a group of keyword texts T [xbmc,antiquated,ffmpeg,atv2,xbmc,ffmpeg,libavcodec,mpeg4video,...] is obtained.
[0127] 3.2.4. Statistics. Count the number of times each keyword appears in T and get the keyword list K [(ffmpeg:14), (xbmc:8), (tomasen:1), ...].
[0128] 3.2.5. Weighting. In this embodiment, the weight W of keyword k is k Defined as
[0129] W k =C k ×P
[0130] Among them, C k is the number of times the keyword appears, and P is an adjustment coefficient. In this embodiment, the name of the open source project is used as an important key information to determine the source of the input source code. Therefore, if the keyword appears in the name of the open source project from each source, P = 1.2, otherwise P = 1. Finally, the keyword dictionary D[(ffmpeg:16.8),(xbmc:9.6),(tomasen:1),...] is obtained.
[0131] 4. Rate the sources.
[0132] 4.1. Scoring the key information of the source. The results are shown in Table 2.
[0133] 4.1.1. Scoring the key information of the source text. Calculate the key information score of each source text according to formula (1).
[0134] 4.1.2. Scoring the non-text key information of the source. In this embodiment, the non-text key information of the source is the number of Stars of the open source project, and its score is defined as the number of Stars itself.
[0135]
[0136]
[0137] Table 2 Text key information scoring
[0138] 4.2. Scoring the sources. The score of each source is calculated according to Formula 2. In this embodiment, it is considered that the name of the open source project is more important than other key information in determining the source of the input source code. Therefore, for the name of the open source project, W i =1.2, other key information W i = 1. The scoring results of each source are shown in Table 3.
[0139] id Open source project name Open Source Project Source Rating 1 xbmc-antiquated 2.390 2 FFMpeg 2.031 3 atv2 1.705 4 FFmpeg 3.662 5 EasyTomato 0.931 6 xbmc-rbp 2.129 7 plex-linux 1.279 8 asus-rt-n66u-merlin 1.187 9 dsl-n55u-bender 1.102 10 DreamShell 1.322 11 splayer 1.007 12 rt-n56u 1.132
[0140] Table 3 Source ratings
[0141] 5. Select the source of the input source code. According to the scoring results obtained in 4.2, the method determines that the source FFmpeg is the source of the input source code. This result is consistent with the actual situation.
[0142] If the traditional time-based open source source determination method is used, the earliest open source project is xbmc-antiquated, which is inconsistent with the actual situation. Compared with the traditional method of determining the source of open source, this method determines the source of open source more accurately.
[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the field can easily think of various equivalent modifications or substitutions within the technical scope disclosed by the present invention, and these modifications or substitutions should all be covered within the protection scope of the present invention.
Claims
1. A method for determining the source of open source code of aviation airborne software, characterized in that: The following steps are involved: Step 1: Search the open source code repository to obtain all sources of input source code; Step 2: Extract key information from each source from the search results of step 1, and construct a keyword dictionary of the key information of the text; Step 3: Based on the key information extracted in step 2 and the keyword dictionary constructed, score the source; Step 4: Based on the scoring result of step 3, select the source of the input source code.
2. The method for determining the source of open source code of aviation airborne software according to claim 1, characterized in that: In step 1, the open source code base is a storage containing all open source codes that are search objects, and its coverage depends on specific needs; the specific form of the open source code base is a database or file storage plus a text search engine; Step 1 specifically includes the following steps: Step 1.1 Open source code feature extraction: First, extract features from the source code in the code repository and map the code into a feature vector representation; Step 1.2 uses the extracted code feature vectors to calculate the similarity between different code repositories; Step 1.3: Filter all possible open source sources of the input source code based on the similarity calculation results.
3. The method for determining the source of open source code of aviation airborne software according to claim 2, characterized in that: In step 1.1, the code is mapped to a feature vector representation using a bag-of-words model or an n-gram model. The bag-of-words model represents each text as a vector of fixed length, and each dimension in the vector represents a word in the vocabulary, the number of occurrences in the text, or other statistical values. The n-gram model represents the text as a sequence of n adjacent words. The specific process of mapping the code to a feature vector representation using the bag-of-words model or the n-gram model is as follows: (1) Data preprocessing: Clean, segment, and remove stop words from the original text to obtain a set of texts; (2) Constructing a vocabulary: Construct a vocabulary of all words that appear in the text set, recording the number of times each word appears in the text set or other statistical values. Other statistical values refer to the part of the source meta-information that can identify the source characteristics; (3) Feature representation: construct feature vectors based on the vocabulary and use the bag-of-words model or n-gram model to represent the feature vectors; (4) Feature selection: Use feature selection methods to select features that can effectively distinguish different categories; (5) Feature scaling: scaling feature vectors to the same scale; (6) Train the classifier: Use the feature vector to train the classifier.
4. The method for determining the source of open source code of aviation airborne software according to claim 3, characterized in that: Step 1.2, use the cosine similarity or Jaccard similarity calculation method to calculate the similarity between different code repositories.
5. The method for determining the source of open source code of aviation airborne software according to claim 4, characterized in that: In step 2, the key information of the source refers to the part of the source's meta-information that can identify the source's characteristics, including but not limited to the name of the open source project, the name of the open source project developer or development team, the creation time of the open source project, the popularity of the open source project, and the file path of the input source code in the open source project; Text key information refers to the key information of the source, the content of which is the key information of the text, including but not limited to the name of the open source project, the name of the open source project developer or development team, and the file path of the input source code in the open source project; The keyword dictionary refers to the keyword information left after text processing of the key information of the text, and its content includes keywords and their weights.
6. The method for determining the source of open source code of aviation airborne software according to claim 5, characterized in that: The steps to build a keyword dictionary include: 2.1 Summarize: combine the key information of the text from all sources; 2.2 Text processing: Format and filter the key information of the text summarized in 2.1, including the following steps: 2.2.1 Formatting: Unify the format of key information by cutting words, removing separators, and converting uppercase and lowercase characters; 2.2.2 Filtering: Remove words that cannot identify source characteristics from the key information of the text; 2.3 Statistics: The words left after text processing are used as keywords, and the number of times each keyword appears is counted; 2.4 Weighting: Assign a weight to each keyword; the weight algorithm is based on the following principles and is designed according to specific needs. The weight is based on the number of occurrences. The more times a keyword appears, the greater its weight. The weight takes into account the importance of the key information in the text where the keyword is located in determining the source of the input source code. The more important the key information in the text, the greater the weight of the keyword appearing in it.
7. The method for determining the source of open source code of aviation airborne software according to claim 6, characterized in that: Step 3 specifically includes the following steps: Step 3.1: Score the key information of the source, including the following steps: Step 3.1.1 Score the key information of the source text; Step 3.1.2: Score the non-textual key information of the source; Step 3.2: Score the sources.
8. The method for determining the source of open source code of aviation airborne software according to claim 7, characterized in that: Step 3.1.1 Score the key information of the source text in the following way: The score S of a key information of a text in the source i Defined as S i = ∑ k k∈K C k × W k (1) Among them, K is the keyword set in the key information, k is the keyword in the set, and C k is the number of times the keyword appears in the key information of the text, W k is the weight of the keyword stored in the keyword dictionary, that is, the score of the key information of the text is the weighted sum of the number of occurrences of each keyword in the key information, and the weight is the weight of the corresponding keyword in the keyword dictionary; In step 3.1.2, the scoring rule for non-text key information follows the principle that the more it reflects the source, the higher the score.
9. The method for determining the source of open source code of aviation airborne software according to claim 8, characterized in that: Step 3.2 Score the sources by: Source rating S o Defined as: S o =∑ i i∈I S i / Max(S i ) × W i (2) Where I is the key information set of the source, i is the key information in the set, S i is the score of the key information, Max(S i ) is the S of each source i The maximum value, W i is the weight of the key information, that is, the score of the source is the weighted sum of the normalized scores of each key information of the source; the weight of the key information varies according to different requirements. The more important the key information is for determining the source of the input source code, the higher its weight. According to the calculation formula, the more complete the key information of the source is, the higher its score is.
10. The method for determining the source of open source code of aviation airborne software according to claim 9, characterized in that: Step 4 is to use the source with the highest score as the source of the input source code.
Citation Information
Patent Citations
Document keyword extraction method based on hypergraph random walk
CN114065758A
Text classification method based on keyword extraction and attention mechanism
CN115168580A
Method and system for extraction of key-terms and synonyms for the key-terms
US20210304749A1
KR20210039136A