Code library statistics and analysis system and method based on GitLab

Abstract

The invention relates to the technical field of software development and version control, and provides a code library statistics and analysis system and method based on GitLab. The system comprises: a Gitlab management module, which is responsible for packaging a Gitlab Api interface and interacting with the Gitlab to realize code statistics and analysis functions; the data acquisition module is responsible for acquiring data in the GitLab code library; the data analysis module is responsible for performing multi-dimensional statistics and analysis on the collected data; the database module is responsible for storing key data of code statistics and analysis; the timed scheduling module is responsible for global scheduling of code statistics and analysis tasks; the Web interface module is responsible for visually displaying and interacting code statistics and analysis results; in addition, the system also comprises a back-end interface service module. According to the method, the API and the function of the GitLab are deeply integrated, comprehensive and deep statistics and analysis are carried out on the codes in the GitLab code library, and high-quality code insight and project management support can be provided for a software development team.

Description

Technical Field

[0001] The present invention relates to the technical field of software development and version control, and in particular to a code base in-depth statistics and analysis system and method for the GitLab platform. The system deeply integrates GitLab's API and functions to conduct comprehensive and in-depth statistics and analysis of the code in the GitLab code base, providing high-quality code insight and project management support for software development teams. Background Art

[0002] As the scale of software development projects continues to expand, the management and maintenance of code bases has become increasingly complex. As a widely used code hosting and version control system, GitLab provides basic code management functions, but it still has many shortcomings in code statistics and analysis, such as single statistical dimensions, insufficient analysis depth, and poor visualization effects. These problems limit the development team's comprehensive understanding and optimization capabilities of the code base.

[0003] Therefore, developing a system that can deeply mine GitLab code repository data, provide multi-dimensional, high-precision statistics and analysis, and have multiple deployment methods to meet the needs of different scenarios has important application value. Summary of the invention

[0004] In view of the above problems, the present invention provides a code library statistics and analysis system and method based on GitLab.

[0005] The technical solution adopted by the present invention is as follows:

[0006] A code base statistics and analysis system based on GitLab, which includes:

[0007] The Gitlab management module is responsible for encapsulating the Gitlab API interface and interacting with GitLab to implement code statistics and analysis functions;

[0008] The data collection module is responsible for collecting data from the GitLab code repository;

[0009] The data analysis module is responsible for multi-dimensional statistics and analysis of the collected data;

[0010] Database module, responsible for storing key data for code statistics and analysis;

[0011] The timing scheduling module is responsible for the global scheduling of code statistics and analysis tasks;

[0012] The web interface module is responsible for visualizing and interacting with the results of code statistics and analysis.

[0013] The back-end interface service module serves as the interactive interface between the back-end and the front-end to realize two-way data transmission.

[0014] Furthermore, the GitLab code repository adopts a centralized deployment method or a multi-point deployment method.

[0015] Furthermore, the data collection module receives a data collection command, communicates with the Gitlab management module according to the command content, and calls the abstract interface provided by the Gitlab management module to obtain the Gitlab original data, and encapsulates the Gitlab original data and stores it in the database.

[0016] Furthermore, the data acquisition module is made extensible and customizable through configuration files and SPI technology; the address and key information of the Gitlab library are defined in the configuration file, each configuration file has a unique id for distinguishing different Gitlab libraries, and all configuration files are automatically scanned when the system starts, and a Gitlab configuration object is generated; a universal adapter and a custom adapter are integrated in the system, and the universal adapter is used to handle Gitlab data acquisition operations in general. If the universal adapter cannot meet the data access requirements, the custom adapter is automatically loaded in SPI mode to realize the data acquisition operation.

[0017] Furthermore, the data analysis module performs the following operations:

[0018] Clean the collected GitLab raw data, remove noise and duplicates, and convert it into a format suitable for analysis;

[0019] Aggregate data according to analysis requirements and count relevant data according to different dimensions;

[0020] Through longitudinal comparison, valuable features for analysis are extracted from the original data, and data in six dimensions are sorted out and stored in different database tables according to the data type.

[0021] Furthermore, the data of the six dimensions include:

[0022] 1) Code Contributors: including committer statistics and contributor activity. The committer statistics include the number of commits, the number of lines of code submitted, and the number of comments for each developer. The contributor activity is to evaluate the activity and continuous contribution of each developer to the project by counting the frequency of each developer's commits.

[0023] 2) Code volume: including total code volume, added / deleted code volume, and comment ratio;

[0024] 3) Projects: including project activity and project code volume distribution; the project activity is to evaluate the project activity and development progress by counting the number of submissions, branches, and merge requests of each project; the project code volume distribution is to analyze the code volume distribution between different projects or modules to understand the scale and complexity of the project;

[0025] 4) Time dimension: including historical trends and statistics for specific time periods; the historical trends are statistics of code submissions and code volume changes in chronological order, displayed in the form of charts to observe the historical development trend of the project; the specific time period statistics are statistics for a certain time period to understand the code contribution within that time period;

[0026] 5) File type: including file type distribution, the file types include source code files, configuration files, and documents;

[0027] 6) Branches and merge requests: including branch statistics and merge requests; the branch statistics include the number of branches in the code base and the number of submissions for each branch; the merge requests include the number of merge requests and the processing time.

[0028] Furthermore, the visual display and interaction provided by the Web interface module include:

[0029] Design an intuitive and easy-to-use user interface, and use rich visual charts to display statistical and analytical results;

[0030] Provides interactive data exploration capabilities, allowing users to freely filter, sort, and compare data, and deeply explore valuable information in the code base;

[0031] Supports data export and sharing functions to facilitate communication and collaboration within the team and between external partners.

[0032] Furthermore, the Gitlab management module is also responsible for processing webhook callback requests and maintaining the connection status with the Gitlab server.

[0033] A GitLab-based code base statistics and analysis method using the above system includes the following steps:

[0034] Initiate the full data processing process through manual clicks by users or timed task triggers;

[0035] After receiving the statistical request, the backend interface service module first records a log and then sends the statistical task to the data collection module;

[0036] The data collection module analyzes statistical tasks and maintains the operation of tasks, tracks task status, and updates log status in real time. The data collection module interacts with Gitlab through the Gitlab management module and obtains data, analyzes Gitlab data, and stores it in the database.

[0037] After data collection is completed, the backend interface service module is notified, indicating that the collection task is completed;

[0038] The backend interface service module starts the data analysis module to start analyzing data;

[0039] The data analysis module analyzes Gitlab data and saves the analysis results to the database.

[0040] Furthermore, the above method uses the following steps to process incremental data:

[0041] The callback address in the Webhook is triggered when the project developer submits or merges code;

[0042] After receiving the request, the callback interface forwards the incremental task to the Gitlab management module;

[0043] The Gitlab management module directly calls the data analysis module;

[0044] The data analysis module starts the analysis task and saves the analysis results to the database.

[0045] The beneficial effects of the present invention are as follows:

[0046] The present invention can deeply mine GitLab code base data, provide multi-dimensional, high-precision code base statistics and analysis, can accurately count the amount of code development of individuals or teams, avoid artificial fraud, and improve the accuracy of statistics. Through automated data collection and processing mechanisms, the present invention can accurately track every change in the code base, including key activities such as code submission, branch creation and merging, and problem tracking, thereby ensuring the comprehensiveness and accuracy of the data. More advanced is that the present invention presents complex code base statistical information to users in the form of charts, reports, etc. through an intuitive visual interface, greatly improving the readability and ease of use of information, and has a variety of deployment methods, which can meet the needs of different scenarios and has important application value. The focus of the present invention is on system architecture, data processing, and front-end display can be custom developed. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is an architecture diagram of the GitLab-based code library statistics and analysis system of the present invention.

[0048] Figure 2It is a flowchart of the full data processing in the data collection and analysis process.

[0049] Figure 3 It is a flowchart of incremental data processing in the data collection and analysis process.

[0050] Figure 4 This is a single system deployment structure diagram.

[0051] Figure 5 This is a diagram of a multi-site deployment structure. DETAILED DESCRIPTION

[0052] The present invention is further described in detail below through specific embodiments and drawings.

[0053] The present invention proposes a code base deep statistics and analysis system and method based on GitLab, which mainly includes the following key technologies and methods:

[0054] 1. Multiple deployment methods:

[0055] Gitlab code repositories may be distributed in multiple locations, and the networks between multiple locations may be isolated from each other. For such complex scenarios, this system supports centralized deployment and multi-point deployment. The final statistical data is aggregated in one of the systems for easy retrieval.

[0056] 2. Multi-database data collection:

[0057] One system supports the collection of multiple Gitlab repositories and supports flexible integration of customized data collection solutions through simple configuration.

[0058] 3. Data collection and analysis:

[0059] 1) Through the GitLab API, collect all the data in the GitLab code repository in real time or regularly, including project information, branch information, submission records, merge requests, code files, etc.

[0060] 2) Support custom Webhook configuration to ensure that the system can respond immediately and perform corresponding statistics when events such as code submission and merge requests occur. Webhook is a message push mechanism provided by Gitlab. In Gitlab's engineering projects, using Webhook allows developers to trigger a pre-configured URL when performing operations such as pushing code and changing project files. Whether pushing code or merging code, users can customize it in GitLab. GitLab will send an http request to the set Webhook URL. In general, Webhook refers to a user-defined HTTP callback. After the event is triggered, GitLab sends an HTTP request. It can be set for each project in Gitlab. The setting method is: Project Settings→Webhooks.

[0061] 3) Design an efficient data analysis engine to conduct in-depth analysis of the collected data and extract key statistical indicators such as the number of code lines, code type, author contribution, submission frequency, file changes, etc.

[0062] 4. Multi-dimensional statistics and analysis:

[0063] The present invention provides a rich set of statistical dimensions, including the following six dimensions:

[0064] 1) Code Contributors:

[0065] Committer statistics: Count the number of commits, lines of code committed (including added and deleted code), number of comments, etc. for each developer. This helps identify core contributors and active developers in the team.

[0066] Contributor activity: By counting the submission frequency of each developer, you can evaluate their activity and continued contribution to the project.

[0067] 2) Amount of code:

[0068] Total Code: Counts the total number of lines of code in the code base, including all files.

[0069] Code addition / deletion: Counts the number of new and deleted lines of code in the code base, which helps understand the growth or refactoring of the code base.

[0070] Comment ratio: Count the ratio of the number of comment lines in the code to the total number of code lines to evaluate the maintainability and readability of the code.

[0071] 3) Project Dimension:

[0072] Project activity: Evaluate the project activity and development progress by counting the number of submissions, branches, merge requests, and other indicators of each project.

[0073] Project code volume distribution: Analyze the code volume distribution between different projects or modules to understand the scale and complexity of the project.

[0074] 4) Time dimension:

[0075] Historical trends: Statistics on code submissions, code volume changes, etc. are collected in chronological order and displayed in charts, which helps to observe the historical development trends of the project.

[0076] Statistics for a specific time period: Statistics are collected for a certain time period (such as a month or a quarter) to understand the code contributions within that time period.

[0077] 5) File Type:

[0078] File type distribution: Count the number and proportion of different file types (such as source code files, configuration files, documents, etc.) in the code base to understand the composition structure of the project.

[0079] 6) Branches and merge requests:

[0080] Branch statistics: Count the number of branches in the code base, the number of commits for each branch, etc. to understand the branch management of the project.

[0081] Merge Requests: Count the number of merge requests, processing time, etc. to evaluate the efficiency and process of team collaboration.

[0082] The present invention supports the generation of detailed statistical reports and analysis results, including code volume overview, contributor analysis, code activity analysis, code quality assessment, etc., providing comprehensive support for project management.

[0083] 5. Visual display and interaction:

[0084] 1) Design an intuitive and easy-to-use user interface, and use rich visual charts (such as line charts, bar charts, heat maps, relationship diagrams, etc.) to display statistical and analysis results.

[0085] 2) Provide interactive data exploration functions, allowing users to freely filter, sort, and compare data, and deeply explore valuable information in the code base.

[0086] 3) Support data export and sharing functions to facilitate communication and collaboration within the team and between external partners.

[0087] The present invention focuses on explaining the statistical analysis method, and the front-end display can be flexibly displayed according to actual conditions.

[0088] Figure 1This is a system architecture diagram of the GitLab-based code base statistics and analysis system of the present invention. The present invention splits the core functions such as code statistics into independent modules to achieve modular development and deployment. Figure 1 As shown, the system includes the following modules:

[0089] 1) Web interface module: provides an intuitive and easy-to-use web interface that supports users to submit code, view statistical information, configure project management, and other operations.

[0090] 2) Gitlab management module: As the proxy layer for interacting with GitLab, it encapsulates Gitlab's processing commands and access control commands, and is an important support for the code statistics function.

[0091] 3) Timing scheduling module: responsible for the global scheduling of statistical tasks and can freely control the scheduling timing.

[0092] 4) Database module: responsible for storing key data such as users, projects, and code statistics, using high-performance database technology to ensure fast access and persistent storage of data.

[0093] 5) Data acquisition module: The core module in the system, responsible for the collection and analysis of statistical data, and relied upon by other modules.

[0094] 6) Data analysis module: The core module in the system, responsible for the analysis of statistical data and relied upon by other modules.

[0095] 7) Back-end interface service module: Serves as the interactive interface between the back-end and the front-end to realize two-way data transmission.

[0096] Figure 2 and Figure 3 This is a flowchart of data collection and analysis. Data collection and analysis are divided into two modes: full and incremental. All operations involved are performed asynchronously. Figure 2 This is the full data processing flow chart. Figure 3 Incremental data processing flow chart.

[0097] like Figure 2 As shown in the figure, the full data processing flow includes the following steps:

[0098] Step 1: The full data processing process can be initiated by a user's manual click or a scheduled task trigger, as shown in step 1 in the figure.

[0099] Step 2: After receiving the statistics request, the backend interface service module first records a log, such as Figure 2 2.1 in the figure is convenient for checking historical statistical records in the future, and then sending the task to the data collection module, such as Figure 22.2 in this section. This module (backend interface service module) is the entrance to the entire system. The main purpose of making this module independent in the design is to facilitate the expansion and development of requests in the future.

[0100] Step 3: After receiving the statistical task, the data collection module needs to parse the task, including clarifying the Gitlab address, which projects to analyze, maintaining the operation of the task, tracking the task status, and updating the log status in step 2.1 in real time. The most important thing in this module is to interact with Gitlab and obtain data, parse Gitlab project data, and store it in the database. Since the interaction with Gitlab involves calling multiple APIs, if the data collection module directly calls the Gitlab API, it will increase the coupling and make it difficult to maintain. Therefore, the Gitlab management module is added. All operations related to Gitlab are completed by this module, and an interface unrelated to the main system business is provided to the outside world.

[0101] Step 4: This step occurs during the entire process of Gitlab data collection by the data collection module, and saves the original data of the Gitlab project into the database.

[0102] Step 5: Data collection is the basic operation of the system. After the collection is completed, the back-end interface service module is notified, indicating that the collection task is completed.

[0103] Step 6: The backend interface service module starts the data analysis module to analyze the data.

[0104] Step 7: The data analysis module analyzes the Gitlab raw data and saves the analysis results to the database.

[0105] Incremental data processing corresponds to the Gitlab webhook function. In the webhook settings of the Gitlab project, configure the callback URL as the system's external service address. Figure 3 As shown, the incremental data processing flow includes the following steps:

[0106] Step 1: When the project developer submits or merges code, the callback address in the webhook is triggered.

[0107] Step 2: After receiving the request, the callback interface forwards the incremental task to the Gitlab management module.

[0108] Step 3: Since the callback request contains the changed project data, there is no need to pull it through the data collection module, and the data analysis module can be called directly.

[0109] Step 4: The data analysis module starts the analysis task and saves it to the database.

[0110] The present invention supports centralized deployment and multi-point deployment of Gitlab code base, and the final statistical data is collected in one system for easy retrieval. Figure 4 This is a single system deployment structure diagram. This deployment structure is to deploy a system (i.e., the GitLab-based code base statistics and analysis system of the present invention) at a designated location, and adapt the Gitlab library at each location through the plug-in configuration method in the system. It is abstracted as a unified collection server layer at the system level. A general adapter is integrated in the system to handle Gitlab data collection operations under general circumstances. A custom adapter interface is provided under the general adapter to handle situations that cannot be met under general circumstances. Users can develop suitable adapters according to their own circumstances. Figure 5 This is a multi-location deployment structure diagram. This deployment method is designed to cope with the scenario of multiple departments distributed in different locations. In order to illustrate the overall structure of multi-location deployment, the system in the figure is simplified. The internal details are the same as those in the single system deployment diagram. Each system collects data independently, and then the data is aggregated into one of the systems by exporting an Excel file.

[0111] The following is a specific description through an embodiment Figure 1 The implementation methods of each module in the GitLab-based code base statistics and analysis system:

[0112] 1. Gitlab management module

[0113] The Gitlab management module implements six dimensions of statistical analysis by abstracting external services and deeply encapsulating the Gitlab API interface. The Gitlab API used in the present invention is described in a classified form below.

[0114] 1) Session: To access the Gitlab API, you need to bring authentication information. To obtain authentication information, you need to log in to Gitlab as an administrator. On the user management page, create an access token under the administrator user. This token is only displayed once and needs to be saved by yourself. Each subsequent API needs to be placed in the request as a parameter.

[0115] 2) Project: GET / projects. Query all projects. The data obtained by this API is the basic data of the entire statistical function. All functions are based on the data returned by this API. The returned data is a list. Each item in the list is a project information, and each item contains basic information about the project.

[0116] 3) Branches: GET / projects / :id / repository / branches. Query all branches of a specified project. Each project can have multiple branches, and independent statistical data is generated for each branch in the statistics.

[0117] 4) Commit records: GET / projects / :id / repository / commits. Query the commit records of a project. This API returns the commit records of a specified project and branch. The commit records are the most basic statistical data. Each commit record contains information such as the committer, commit time, and the number of lines of code committed.

[0118] 5) Notes: GET / projects / :id / notes. A description of the notes will be generated for each commit of each branch. By counting the notes, you can understand whether the developer's notes are standardized each time they are committed.

[0119] 6) Code merge: GET / projects / :id / merge. Query the branch merge status of the specified project. For each merge, a piece of data will be recorded, including the source branch and the destination branch, the number of merged files, the number of conflicting files, the merger, and other information. This data can be used for statistical analysis of branch merges.

[0120] 7) File information: GET / projects / :id / files. Query the composition of project files under the specified project branch, including the number of files, file names and other information.

[0121] These interfaces cover the data required in 6 dimensions. This Gitlab management module encapsulates the interface data, encapsulates the complex interactions with Gitlab inside the module, and provides a clear add, delete, modify and query interface to the external module. In addition, this module is also responsible for processing webhook callback requests, which need to be configured on the Gitlab management page. The Gitlab management module is responsible for maintaining the connection status with the Gitlab server.

[0122] 2. Data acquisition module

[0123] The main responsibility of the data collection module is to receive data collection commands and communicate with the Gitlab management module according to the command content. The data collection module calls the abstract interface provided by the Gitlab management module to obtain the Gitlab original data, encapsulates the Gitlab original data, and stores it in the database.

[0124] Data collection is an important function for processing data access in the system. In actual application, it will face various environments and may also access multiple code bases, so the data access part needs to be made extensible and customizable when the system is designed. In order to achieve this goal, this system mainly uses configuration files and SPI (Service Provider Interface, a design pattern that provides a service discovery mechanism) technology. The configuration file mainly defines the address and key of the Gitlab library and other information. It is important that each configuration file has a unique id to distinguish different Gitlab libraries. When the system starts, it will automatically scan all Gitlab configuration files and generate Gitlab configuration objects. Since the system is integrated with a universal adapter, the universal adapter provides a Gitlab data acquisition, analysis and storage logic that can meet the needs of most scenarios. If the Gitlab to be connected has no special requirements, just add this configuration file. The system will automatically scan and load the configuration file and create a Gitlab library statistics task template corresponding to this configuration file. It will be automatically called when the user or scheduled task calls the statistics task. If the general adapter cannot meet the data access requirements, the system provides an SPI method to automatically load a custom adapter. The custom adapter is a new project, independent of the system invented here. Put the written custom adapter into the system directory, restart the system and load it, thus realizing the expansion of Gitlab's special connection requirements.

[0125] 3. Data Analysis Module

[0126] The data analysis module analyzes and processes Gitlab's raw data, cleans the collected raw data, removes noise and duplicates, and converts it into a format suitable for analysis. According to the analysis requirements, the data is aggregated, such as statistics on the number of submissions, processing time for merge requests, and problem resolution rate by time, team, project, etc. In terms of feature extraction, features valuable for analysis are extracted from the raw data through longitudinal comparison, such as code change frequency, team member activity, etc., and data in 6 dimensions are sorted out. These analyzed data are stored in different database tables according to type. The following is an explanation of the data analysis for each dimension:

[0127] 1) Code Contributors:

[0128] This dimension is based on people, and analyzes the amount of code submitted and the activity of each person. The statistical data for this dimension is based on the data of the project submission record interface in the Gitlab API, which includes information about the submitter, submission time, and number of lines of code. The individual submission volume indicator can be used to aggregate the submission volume by person. The activity of the submitter can be calculated by grouping the submission time by time period and calculating the number of submissions by the submitter in each time period.

[0129] 2) Amount of code:

[0130] This dimension is based on projects. The total code volume is the summary of the code line number field in all commit record data in each project. The added / deleted code volume is the summary of the corresponding added / deleted code lines in all commit record data. The comment ratio is an instantaneous indicator. The project file content can be obtained through the file information interface of the Gitlab API. The comment ratio of a project is determined by the ratio of the total number of bytes occupied by the comment information in the file content to the total number of bytes in the file.

[0131] 3) Project Dimension:

[0132] Project activity is mainly based on project submission record data, calculating the number of submissions of a project in a time period. It is a different dimension from the submitter activity. The project code volume distribution also counts the total code volume of each project on a project basis, and the data is also based on submission record data.

[0133] 4) Time dimension:

[0134] The historical trend in the time dimension is based on the project activity data that has been counted in the project dimension data. The number of lines of code in each time period is recorded in chronological order, which helps to observe the historical development trend of the project.

[0135] 5) File Type:

[0136] For each project, based on the file interface data of the Gitlab API, which contains the file list information of all levels of directories in the project, all files are divided into three types: source code files, configuration files, and documents according to the file suffix. The number of each type of file is calculated and saved in the database.

[0137] 6) Branches and merge requests:

[0138] Branch statistics: Count the number of branches in the code base, the number of commits for each branch, etc. to understand the branch management of the project.

[0139] Merge Requests: Count the number of merge requests, processing time, etc. to evaluate the efficiency and process of team collaboration.

[0140] 4. Backend interface service module

[0141] The back-end interface service module is mainly responsible for external request processing, distributing data collection tasks to data collection modules and querying statistical data. The front-end calls the back-end interface service module through HTTP requests, and the back-end interface service module processes these requests, retrieves or updates data from the database or other modules, and returns the results to the front-end. This mechanism realizes the two-way transmission of data, allowing users to view and operate data on the interface.

[0142] 5. Timing Scheduling Module

[0143] The timer scheduling module is responsible for scheduling statistical tasks. In order to achieve near real-time performance of Gitlab statistical data and keep the data in the system relatively up to date, an open source timer scheduling framework is integrated into the system. This framework is started with the system. The timer framework provides a management page, through which the call frequency of the Gitlab statistical service interface is configured. During the implementation process, there are many types of timer frameworks, and developers can use the existing deployed timer scheduling framework according to their own situation.

[0144] It should be understood that the systems and methods disclosed in the above embodiments provided by the present invention can be implemented in other ways. For example, the system embodiments described above are only illustrative, and the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0145] The modules implemented in the form of software function modules in the present invention can be stored in a computer-readable storage medium. The software function unit is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0146] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and implement it accordingly. It can be understood by those skilled in the art that various replacements, changes and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the contents disclosed in the embodiments of this specification, and the scope of protection of the present invention shall be subject to the scope defined in the claims.

Claims

1. A code base statistics and analysis system based on GitLab, characterized in that: include: The Gitlab management module is responsible for encapsulating the Gitlab API interface and interacting with GitLab to implement code statistics and analysis functions; The data collection module is responsible for collecting data from the GitLab code repository; The data analysis module is responsible for multi-dimensional statistics and analysis of the collected data; Database module, responsible for storing key data for code statistics and analysis; The timing scheduling module is responsible for the global scheduling of code statistics and analysis tasks; Web interface module, responsible for visual display and interaction of code statistics and analysis results; The back-end interface service module serves as the interactive interface between the back-end and the front-end to realize two-way data transmission.

2. The system according to claim 1, characterized in that The GitLab code repository adopts a centralized deployment method or a multi-point deployment method.

3. The system according to claim 1, characterized in that The data collection module receives the data collection command, communicates with the Gitlab management module according to the command content, calls the abstract interface provided by the Gitlab management module to obtain the Gitlab original data, and encapsulates the Gitlab original data and stores it in the database.

4. The system according to claim 3, characterized in that The data acquisition module is scalable and customizable through configuration files and SPI technology; the address and key information of the Gitlab library are defined in the configuration file, each configuration file has a unique id for distinguishing different Gitlab libraries, and all configuration files are automatically scanned when the system starts, and a Gitlab configuration object is generated; a universal adapter and a custom adapter are integrated in the system, and the universal adapter is used to handle Gitlab data acquisition operations in general. If the universal adapter cannot meet the data access requirements, the custom adapter is automatically loaded in SPI mode to realize the data acquisition operation.

5. The system according to claim 1, characterized in that The data analysis module performs the following operations: Clean the collected GitLab raw data, remove noise and duplicates, and convert it into a format suitable for analysis; Aggregate data according to analysis requirements and count relevant data according to different dimensions; Through longitudinal comparison, valuable features for analysis are extracted from the original data, and data in six dimensions are sorted out and stored in different database tables according to the data type.

6. The system according to claim 1, characterized in that The data of the six dimensions include: 1) Code Contributors: including committer statistics and contributor activity. The committer statistics include the number of commits, the number of lines of code submitted, and the number of comments for each developer. The contributor activity is to evaluate the activity and continuous contribution of each developer to the project by counting the frequency of each developer's commits. 2) Code volume: including total code volume, added / deleted code volume, and comment ratio; 3) Projects: including project activity and project code volume distribution; the project activity is to evaluate the project activity and development progress by counting the number of submissions, branches, and merge requests of each project; the project code volume distribution is to analyze the code volume distribution between different projects or modules to understand the scale and complexity of the project; 4) Time dimension: including historical trends and statistics for specific time periods; the historical trends are statistics of code submissions and code volume changes in chronological order, displayed in the form of charts to observe the historical development trend of the project; the specific time period statistics are statistics for a certain time period to understand the code contribution within that time period; 5) File type: including file type distribution, the file types include source code files, configuration files, and documents; 6) Branches and merge requests: including branch statistics and merge requests; the branch statistics include the number of branches in the code base and the number of submissions for each branch; the merge requests include the number of merge requests and the processing time.

7. The system according to claim 1, characterized in that The visual display and interaction provided by the Web interface module include: Design an intuitive and easy-to-use user interface, and use rich visual charts to display statistical and analytical results; Provides interactive data exploration capabilities, allowing users to freely filter, sort, and compare data, and deeply explore valuable information in the code base; Supports data export and sharing functions to facilitate communication and collaboration within the team and between external partners.

8. The system according to claim 1, characterized in that The Gitlab management module is also responsible for processing webhook callback requests and maintaining the connection status with the Gitlab server.

9. A code base statistics and analysis method based on GitLab using the system according to any one of claims 1 to 8, characterized in that: The following steps are involved: Initiate the full data processing process through manual clicks by users or timed task triggers; After receiving the statistical request, the backend interface service module first records a log and then sends the statistical task to the data collection module; The data collection module analyzes statistical tasks and maintains the operation of tasks, tracks task status, and updates log status in real time; The data collection module interacts with Gitlab through the Gitlab management module to obtain data, parses Gitlab data and stores it in the database; After data collection is completed, the backend interface service module is notified, indicating that the collection task is completed; The backend interface service module starts the data analysis module to start analyzing data; The data analysis module analyzes Gitlab data and saves the analysis results to the database.

10. The method according to claim 9, characterized in that The following steps are used to process incremental data: The callback address in the Webhook is triggered when the project developer submits or merges code; After receiving the request, the callback interface forwards the incremental task to the Gitlab management module; The Gitlab management module directly calls the data analysis module; The data analysis module starts the analysis task and saves the analysis results to the database.

Images