A method of batch software automated build

Abstract

The application discloses a kind of batch software automatic construction method, builds software on Linux and Kylin system, specifically includes the following steps: using AI neural network model analysis each source code description file of software, extracts the information of dependent installation, configuration or compilation;Write regular expression to extract the information of dependent installation, configuration or compilation in each source code description file of software;Contrast the information obtained by the above two methods, make a comprehensive decision;If the relevant information cannot be obtained by using the above two methods, then traverse the configuration file information in the first folder of source code directory by the method of traversal, select the appropriate default compilation method;Grab the information printed when compiling, use regular method to grab key flag information, judge whether the compilation is successful or not.By the application, a general method is established, which greatly simplifies the software construction work in the daily work of developers, and can quickly perform the construction work of a large number of code packages.

Description

Technical Field

[0001] This invention relates to the field of software building technology for LINUX and Kylin systems, specifically a method for automated batch software building. Background Technology

[0002] Currently, operating systems are crucial to national security. Domestic operating systems, represented by Kylin, have developed over many years and have released several relatively mature products. However, their market share in the civilian sector remains low. The main reason for this is that Kylin started later than Windows and its ecosystem is not yet fully developed. To expand the ecosystem of Kylin and Linux software, plans are underway to acquire software from domestic and international open-source repositories and conduct adaptation work. Software building is a vital part of software development, and it often requires significant time, especially when building batches of software. Therefore, simplifying software building is not only necessary to streamline development work but also crucial for rapidly iterating software products to quickly adapt to operating systems and be launched to the market. Summary of the Invention

[0003] The purpose of this invention is to provide a method for automated batch software construction to solve the problem of long time consumption in existing batch software construction technologies.

[0004] This invention discloses a method for automated batch software building, which builds software on Linux and Kylin systems, specifically including the following steps:

[0005] S1. Analyze the source code documentation of the software using an AI neural network model to extract information about dependencies, installation, configuration, or compilation.

[0006] S2. Write regular expressions to extract dependency installation, configuration, or compilation information from the source code documentation files of the software;

[0007] S3. Compare and analyze the information obtained by using AI neural network models and writing regular expressions, and make a comprehensive decision. If neither of these two methods can obtain relevant information, then traverse the configuration file information in the first folder of the source code directory and select the appropriate default compilation method.

[0008] S4. Based on the information obtained by using AI neural network models, writing regular expressions, and traversing methods, configure and compile the source code, capture the information printed during compilation, use regular expression methods to capture key flag information, and determine whether the compilation is successful.

[0009] The specific process of step S1 is as follows:

[0010] S11. Manually select, clean, and label the dataset: Manually select source code documentation files containing information about dependency installation, configuration, or compilation from the software's source code repository as the dataset, and label each piece of information in the dataset.

[0011] S12. An optimized network structure combining encoding, Transformer Encoder, classification, and loss optimization is adopted, and a BERT pre-trained model is loaded as the AI ​​neural network model.

[0012] S13. Using the source code documentation within the dataset, train and optimize the AI ​​neural network model based on the model loss function.

[0013] S14. In the model prediction stage, the source code documentation to be analyzed is segmented into sentences. The trained AI neural network model is used to obtain the score of each sentence. The sentences are sorted from high to low according to the scores and then merged with the results of regular expression extraction in the next step. The sentences are selected as the summary.

[0014] The specific steps of step S11 are as follows:

[0015] Step S111: Select source packages from the source code repository that contain information about dependency installation, configuration, or compilation, and clean out the source packages that do not meet the requirements.

[0016] Step S112: Save the description file of the selected source code package that meets the requirements, and insert the CLS mark at the beginning of each sentence and the SEP mark at the end of each sentence according to the input text format.

[0017] In step S13, the model loss function is the binary cross-entropy loss function.

[0018] In step S2, the regular expression uses the re.findall function to capture information about the dependencies, installation, configuration, or compilation in the source code documentation files of the software.

[0019] The beneficial effects of this invention are as follows:

[0020] This invention provides an automated software build method that utilizes AI text analysis of documentation on Linux and Kylin systems to extract information such as dependency installation, configuration, and compilation. It then employs regular expressions to obtain this information, compares and analyzes the two extraction results, and makes a comprehensive decision. If the above methods fail to obtain the information, it iterates through configuration file information in the first-level folder of the source code directory to select the appropriate default compilation option. This invention establishes a universal method that greatly simplifies the software build process for developers in their daily work, enabling the rapid build of large numbers of code packages.

[0021] Tests have shown that the software automation construction method of this invention can automatically complete the construction of batch software, and the software and operating system compatibility efficiency is greatly improved. Attached Figure Description

[0022] Figure 1 This is a flowchart of the method of the present invention;

[0023] Figure 2 This is a structural diagram of the AI ​​neural network model of the present invention;

[0024] Figure 3 This is a summary of the default configuration compilation commands for this invention;

[0025] Figure 4 This is a structural diagram of the BERT pre-trained model. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] like Figures 1-4 As shown, the present invention provides a method for automated batch software building, which builds software on Linux and Kylin systems, specifically including the following steps:

[0028] S1. Analyze the source code documentation of the software using an AI neural network model to extract information about dependencies, installation, configuration, or compilation.

[0029] S2. Write regular expressions to extract dependency installation, configuration, or compilation information from the source code documentation files of the software. Based on past manual compilation practices, the dependency installation, configuration, and compilation commands in the documentation files have relatively fixed formats. Therefore, write regular expressions to extract dependency installation, configuration, and compilation information from the documentation files. Specifically, the regular expression can use the `re.findall` function to extract this information from the software's source code documentation files.

[0030] The regular expression code is as follows:

[0031] apt_command=re.findall(r'sudo apt([^\<]+)',lines[i])

[0032] cmake_command=re.findall(r'^cmake([^\<]+)',lines[i])

[0033] make_command=re.findall(r'^make([^\<]+)',lines[i]).

[0034] S3. Compare and analyze the information obtained by using AI neural network models and writing regular expressions, and make a comprehensive decision (i.e., first use the command statement containing the crawled information to configure and compile the information obtained by AI neural network models and writing regular expressions; if the configuration fails, then use the command statement containing the crawled information to configure and compile the information); if neither AI neural network model nor writing regular expressions can obtain relevant information, then traverse the configuration file information in the first folder of the source code directory and select the appropriate default compilation method for configuration and compilation, that is, select the default compilation method corresponding to the crawled information for configuration and compilation.

[0035] Since program configuration, compilation, and installation have fixed sensitive words and structures, such as: . / configure, cmake.-B build, make-j, make install, qmake*.pro, ninja build, mvn build, graddle build, gobuild, etc., and sentences have specific starting words, the appropriate statements can be automatically selected by comparing and analyzing the sentences extracted by the AI ​​neural network model with the differences obtained by regular expressions. In this embodiment, the analysis results of the AI ​​neural network model will be adapted with the compilation rules to see if they conform to the command specifications; at the same time, the regular expression crawling method will also crawl the configuration and compilation related information, and compare it with the results extracted by the AI ​​neural network model to see if the results extracted by the AI ​​neural network model contain these commands. If not, they are added to the corresponding command queue.

[0036] If the above method fails to obtain the relevant information, then the configuration file information in the first folder of the source code directory is traversed to select the appropriate default configuration and compilation method. The default method is as follows: Figure 3The following configuration files are shown: those with configure (default command is configure); those with CMakeLists.txt (default command is cmake); those with *.pro files (default command is qmake); those with meson.build files (default command is meson build); those with Makefile files (default command is make); those with build.ninja files (default command is ninja); those with pom.xml files (default command is mvnbuild); those with build.gradle files (default command is gradle build); those with go.deb / go.mod files (default command is go build); and those with Cargo.toml files (default command is Cargo build).

[0037] Taking CMake information capture as an example: First, in the RRADME file under the source code directory, an AI neural network model and regular expressions are used to capture the "cmake" information in the file summary. First, a longer command statement containing the "cmake" command is used for configuration. If the configuration fails, a shorter command statement containing the "cmake" command is used for configuration. If the configuration still fails, the CMakeLists.txt configuration file information in the first-level folder of the source code directory is traversed to determine the default CMake command for configuration. After the configuration command is executed, a Makefile file is generated, and the make command is used for compilation.

[0038] S4. Based on the information obtained by using AI neural network models, writing regular expressions, and traversing methods, configure and compile the source code, capture the information printed during compilation, use regular expression methods to capture key flag information, and determine whether the compilation is successful.

[0039] In this embodiment, the success of compilation is determined by capturing failure information or messages such as success or build target 100%.

[0040] The specific process of step S1 is as follows:

[0041] S11. Manually select, clean, and label the dataset: Manually select source code documentation files containing information about dependency installation, configuration, or compilation from the software's source code library as the dataset, and label each piece of information in the dataset.

[0042] The specific steps of step S11 are as follows:

[0043] Step S111: Select source packages from the source code repository that contain information about dependency installation, configuration, or compilation, and clean out the source packages that do not meet the requirements.

[0044] Step S112: Save the description files of the selected source code packages that meet the requirements. Insert a CLS flag at the beginning of each sentence and a SEP flag at the end of each sentence according to the input text format. Simultaneously, label each sentence with its category as the actual value. In this embodiment, sentences can be labeled with tags such as "dependency installation," "configuration," or "compilation" based on their information content.

[0045] S12. An optimized network structure combining encoding, Transformer Encoder, classification, and loss optimization is adopted, and a BERT pre-trained model is loaded as the AI ​​neural network model.

[0046] like Figure 4 As shown, the encoding layer is Embedding, the Transformer Encoder is the T1, T2... layers after BERT, and then classification prediction is completed. The loss is the comparison between the classification prediction and the true value above as the training loss.

[0047] like Figure 4 As shown, the BERT pre-trained model mainly consists of a sentence encoding layer ( Figure 4 The part above BERT) and the summarization judgment layer ( Figure 4 The document consists of the following parts (below BERT): the sentence encoding layer obtains the sentence vector encoding of each sentence in the document through the upper encoding model, the summary judgment layer selects and judges each sentence through the classification network, scores each sentence, and finally selects the best top-n sentences as the key information of the document.

[0048] like Figure 3 As shown, the specific process of this step is as follows:

[0049] S121, where the encoding transforms large sparse vectors into a low-dimensional space that preserves semantic relationships, using token embeddings and segmentation embeddings (i.e., Figure 3 The sum of Label Embeddings and Position Embeddings in the BERT dataset is used as the input to BERT.

[0050] S122. Input the feature vector output by BERT into the Transformer Encoder, and output the classification and the original sentence, such as... Figure 3 As shown.

[0051] Furthermore, in this embodiment, more positional embeddings are added to the BERT pre-trained model to overcome the original input length limitation (e.g., the original BERT positional encoding length is limited to a fixed value; here, we increase this fixed value, such as by multiplying it by 100, which can result in more encodings). The positional embeddings are randomly initialized and fine-tuned simultaneously with other parameters during training. The network layer parameter values ​​in the encoding (i.e., the parameters mentioned above) are used to calculate the perceptual loss between the output encoding with more embedded parameters and the original output encoding, ensuring that the sentence encoding layer can correctly distinguish between sentences. Fine-tuning is performed based on the original BERT parameters while simultaneously reducing the learning rate.

[0052] In this embodiment, due to the BERT model's MLM pre-training mechanism, its output vector is a vector of each token. Pre-training involves encoding using the parameters provided by the BERT authors, and the token is the result of this encoding. Even though the delimiter can distinguish different input sentences, there are only two labels (sentence A or sentence B), which is significantly different from extractive summarization, which requires separating multiple sentences. Therefore, the input to the BERT model was modified as follows:

[0053] Insert [CLS] and [SEP] tags before and after each sentence in the document, and input the [CLS] tag before each sentence into the output vector of the model as the sentence vector representation. For example: the document is

[0054] “Then, use CMake to build the project

[0055] cmake -B build

[0056] cmake--build build--config RelWithDebInfo-j",

[0057] The input sequence becomes "[CLS]Then, use CMake to build the project[SEP]".

[0058] [CLS]cmake -B build [SEP]

[0059] [CLS]cmake --build build --config RelWithDebInfo-j[SEP]".

[0060] S13. Using the source code documentation within the dataset, train and optimize the AI ​​neural network model according to the model loss function. In this embodiment, after obtaining the sentence vector of each sentence in the document from the sentence encoding layer, a summary judgment layer is constructed to obtain the importance of each sentence under document-level features. For each sentence, the final prediction score is calculated, and the model loss is the binary cross-entropy relative to the label. That is, the model loss function adopts the binary cross-entropy loss function. The binary cross-entropy loss function is:

[0061]

[0062] The binary cross-entropy loss function is an existing technique.

[0063] S14. In the model prediction stage, the source code documentation to be analyzed is segmented into sentences, and a trained AI neural network model is used to obtain the score of each sentence. The sentences are sorted from high to low according to the scores and fused with the regular expression crawling results in the next step. The sentences are selected as the summary. In this embodiment, the sentences with high scores obtained by the AI ​​neural network model are fused with the regular expression crawling results in the next step.

[0064] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method of batch software automation build, characterized by, Building software on Linux and Kylin systems involves the following steps: S1. Analyze the source code documentation of the software using an AI neural network model to extract information about dependencies, installation, configuration, or compilation. S2. Write regular expressions to extract dependency installation, configuration, or compilation information from the source code documentation files of the software; S3. Compare and analyze the information obtained by using AI neural network models and writing regular expressions, and make a comprehensive decision. If neither of these two methods can obtain relevant information, then traverse the configuration file information in the first folder of the source code directory and select the appropriate default compilation method. S4. Based on the information obtained by using AI neural network models, writing regular expressions, and traversing methods, configure and compile the source code, capture the information printed during compilation, use regular expression methods to capture key flag information, and determine whether the compilation is successful. The S3 involves first configuring and compiling the information obtained by the AI ​​neural network model and writing regular expressions using a longer command statement containing the crawling information. If the configuration fails, then a shorter command statement containing the crawling information is used for configuration and compilation.

2. The method of claim 1, wherein, The specific process of step S1 is as follows: S11. Manually select, clean, and label the dataset: Manually select source code documentation files containing information about dependency installation, configuration, or compilation from the software's source code repository as the dataset, and label each piece of information in the dataset. S12. An optimized network structure combining encoding, Transformer Encoder, classification, and loss optimization is adopted, and a BERT pre-trained model is loaded as the AI ​​neural network model. S13. Using the source code documentation within the dataset, train and optimize the AI ​​neural network model based on the model loss function. S14. In the model prediction stage, the source code documentation to be analyzed is segmented into sentences. The trained AI neural network model is used to obtain the score of each sentence. The sentences are sorted from high to low according to the scores and then merged with the results of regular expression extraction in the next step. The sentences are selected as the summary.

3. The method of claim 2, wherein, The specific steps of step S11 are as follows: Step S111: Select source packages from the source code repository that contain information about dependency installation, configuration, or compilation, and clean out the source packages that do not meet the requirements. Step S112: Save the description file of the selected source code package that meets the requirements, and insert the CLS mark at the beginning of each sentence and the SEP mark at the end of each sentence according to the input text format.

4. The method of claim 2, wherein: In step S13, the model loss function adopts the binary cross-entropy loss function.

5. The method of claim 1, wherein: In step S2, the regular expression uses the re.findall function to capture information about the dependencies, installation, configuration, or compilation in the source code documentation files of the software.

Images