Incremental scanning report generation method and device based on SonarQube
By integrating the code warehouse hosting platform and incremental scanning platform on the SonarQube community version, dynamic cropping and mapping relationship construction is solved, the limitations of the SonarQube community version in multi-branch analysis, resource consumption and concurrent processing capabilities are realized, and large-scale projects are realized Fast incremental code scanning and efficient resource utilization improve analysis efficiency and code quality.
Patent Information
- Application Number
- CN202411086653.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-11-26
AI Technical Summary
The SonarQube community version has limitations in multi-branch analysis, resource consumption and concurrent processing capabilities, resulting in slow analysis speed of large projects and low incremental code recognition efficiency, making it difficult to meet customized analysis requirements and hardware resource limitations.
Submit code merge requests through the code warehouse hosting platform, trigger the Webhook mechanism, obtain and crop change files on the incremental scanning platform, use sonarScanner to perform incremental scanning on the SonarQube platform, generate incremental code scanning reports, and build mapping relationships between requirements and static scan results. , dynamic resource provisioning and Myers differential algorithm optimization scanning process.
It solves the problem of slow analysis of large-scale projects, reduces static scanning resource losses, and quickly generates incremental code scanning reports, improves efficiency and ensures that the code quality meets the requirements access control indicators.
Smart Images

Figure CN119025422A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of static code scanning technology, and in particular to a method and device for generating incremental scanning reports based on SonarQube, and in particular to a method and device for quickly generating incremental code scanning reports based on SonarQube community version. Background Art
[0002] Static code scanning, as a quality assurance method, can avoid many low-level errors before testing. For code repositories with a lot of technical debt, the appropriate implementation method is usually to use incremental code scanning to ensure that there are no irregularities in the newly added code, and then gradually complete the technical rectification of the entire code repository.
[0003] As the mainstream code quality management platform in the industry, SonarQube can discover vulnerabilities, code smells, potential errors and non-compliant coding standards in the code through simple rule configuration. The existing SonarQube has four business versions: community version, development version, enterprise version and data center version. Currently, only the community version is free for public use. Although the community version of SonarQube has the advantages of being free and easy to deploy, it also has some limitations, such as: (1) Branch management restrictions. The community version of SonarQube does not support independent analysis of multiple branches, which means that all branches of a project will share the same analysis results, making it difficult to meet customized analysis requirements in different demand scenarios; (2) Resource consumption and performance bottlenecks. As the scale of the project expands, the demand for server resources in the code scanning and analysis process increases significantly. Especially in large projects, the concurrent processing capability of SonarQube is limited, resulting in a significant slowdown in the analysis speed; (3) Incremental code identification mechanism. The community version of SonarQube uses a time-based or before-and-after analysis result comparison method to identify incremental code, which lacks effective association with specific demand changes. In scenarios involving multiple requirements going online at the same time, there may be problems with low analysis efficiency, especially within limited hardware resources and a tight time window, making it a challenge to complete incremental scanning of multiple requirements.
[0004] To this end, the present invention provides a method and device for generating an incremental scanning report based on SonarQube. Summary of the invention
[0005] Based on this, it is necessary to provide a method and device for generating incremental scanning reports based on SonarQube to address the above technical problems.
[0006] According to one aspect of the present invention, a method for generating an incremental scanning report based on SonarQube is provided, which is used for an incremental scanning platform, a code repository hosting platform that interacts with the incremental scanning platform, and a SonarQube platform that interacts with the incremental scanning platform. The method includes: the code repository hosting platform obtains a code merge request submitted by a developer, and triggers a Webhook mechanism to send a code merge request to the incremental scanning platform; the incremental scanning platform receives the code merge request sent by the code repository hosting platform, and obtains the relationship information among the requirement information, code repository information, and code branch information entered by the developer, and creates a corresponding sonarQube project on the SonarQube platform through an API interface and configures the corresponding scanning Rule information, at the same time, builds a mapping relationship between requirement information, code repository information, code branch information, code merge request and sonarQube project, and sends the mapping relationship to the code repository hosting platform; the incremental scanning platform performs code trimming on the changed files in the local code repository according to the code merge request, generates files for static analysis, executes the sonarScanner of the incremental scanning platform, sonarScanner pulls the sonarQube project in the SonarQube platform and triggers scanning, generates scanning information, and sends the scanning information to the SonarQube platform; the SonarQube platform receives the scanning information and generates an incremental code scanning report based on the scanning information.
[0007] In some optional implementations of some embodiments, the code merge request includes demand branch information and online branch information, wherein the demand branch information includes multiple code branch information, and the online branch information includes multiple release branch information.
[0008] In some optional implementations of some embodiments, the incremental scanning platform performs code trimming on the changed files in the local code repository according to the code merge request, and generates files for static analysis, which specifically includes: step one, warehouse pulling, using the JGIT open source library to clone the code repository on the remote code repository hosting platform to the local code repository, and at the same time, switching the local code repository to the code branch; step two, obtaining the changed file path, calling the code repository hosting platform, comparing the API differences between the demand branch and the online branch, obtaining the changed file path, the changed file and its modification type, and generating the corresponding task to be trimmed; step three, dynamic resource allocation, based on thread pool queuing Quantity strategy and file change quantity strategy, obtain the tasks to be trimmed, and dynamically match the corresponding thread pool; Step 4, initial trimming, obtain the tasks to be trimmed in the thread pool, and based on the modification type contained in the tasks to be trimmed, perform initial trimming on the changed files contained in the tasks to be trimmed, retain the modified files with the modification type of new and / or modified, and obtain the initial trimming file; Step 5, secondary trimming, call the demand branch information and the online branch information through the code warehouse hosting platform, use the Myers difference algorithm to calculate the modification parameter data that needs to be modified in the initial trimming file, and perform secondary trimming on the initial trimming file based on the modification parameter data to generate a file for static analysis.
[0009] In some optional implementations of some embodiments, the modification parameter data includes newly added code line number range information, deleted code line number range information and modified code line number range information.
[0010] In some optional implementations of some embodiments, the sonarScanner pulls the sonarQube project in the SonarQube platform and triggers a scan to generate scan information, specifically including: sonarScanner pulls the sonarQube project in the SonarQube platform and triggers a scan, scans the incremental code based on the scan rule information corresponding to the sonarQube project and the incremental report indicator information preset in the incremental code scan report, and during the scanning process, polls to obtain the code scanning access control data contained in the scan rule information, and generates scan information containing multiple incremental report indicator data.
[0011] In some optional implementations of some embodiments, the SonarQube platform receives scanning information and generates an incremental code scanning report based on the scanning information, specifically including: the SonarQube platform receives scanning information, and based on the incremental reporting indicator information preset in the incremental code scanning report, sums multiple incremental reporting indicator data to obtain comprehensive incremental reporting indicator data, compares the comprehensive incremental reporting indicator data with the code scanning access control data, and determines whether the comprehensive incremental reporting indicator data is less than or equal to the code scanning access control data; if the comprehensive incremental reporting indicator data is less than or equal to the code scanning access control data, then generates an incremental code scanning report for releasing the corresponding requirement information; if the comprehensive incremental reporting indicator data is greater than the code scanning access control data, then generates an incremental code scanning report for not releasing the corresponding requirement information.
[0012] In some optional implementations of some embodiments, it also includes: the SonarQube platform is provided with a visual interface for developers to view incremental code scanning reports.
[0013] According to a second aspect of the present invention, a device for generating incremental scanning reports based on SonarQube is provided, which is used for an incremental scanning platform, a code repository hosting platform interacting with the incremental scanning platform, and a SonarQube platform interacting with the incremental scanning platform, including: the code repository hosting platform includes a code merge request module and a request sending module; wherein the code merge request module is used to obtain code merge requests submitted by developers; the request sending module is used to trigger a Webhook mechanism to send code merge requests to the incremental scanning platform; the incremental scanning platform includes a request receiving module, an information entry module, a sonarQube project building module, a mapping relationship building module, a code clipping module and a scanning module; wherein the request receiving module is used to receive code merge requests sent by the code repository hosting platform; the information entry module is used to obtain the relationship information among the requirement information, code repository information, and code branch information entered by the developer; the sonarQube project building module is used to connect to the incremental scanning platform through an API Create a corresponding sonarQube project on the SonarQube platform and configure the corresponding scanning rule information; the mapping relationship construction module is used to construct the mapping relationship between the requirement information, code repository information, code branch information, code merge request and sonarQube project, and send the mapping relationship to the code repository hosting platform; the code trimming module is used to perform code trimming on the changed files in the local code repository according to the code merge request, and generate files for static analysis; the scanning module is used to execute the sonarScanner of the incremental scanning platform, sonarScanner pulls the sonarQube project in the SonarQube platform and triggers scanning, generates scanning information, and sends the scanning information to the SonarQube platform; the SonarQube platform includes an information receiving module and a report generating module; wherein the information receiving module is used to receive scanning information; the report generating module is used to generate an incremental code scanning report based on the scanning information.
[0014] According to a third aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.
[0015] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium storing a computer program, wherein the computer program implements the steps of the above method when executed by a processor.
[0016] The advantages and beneficial effects of the present invention are: the incremental scanning report generation method and device based on SonarQube provided by the present invention, the method submits a code merge request through a code repository hosting platform, and triggers a webhook mechanism to send a code merge request to an incremental scanning platform, the incremental scanning platform performs code clipping on the changed files in the local code repository according to the code merge request, and only retains the incremental code, which can solve the problem of slow full analysis of large projects on the one hand, and effectively reduce the loss of code static scanning resources on the other hand, and quickly generate incremental code scanning reports to reduce the waiting time of development and testing personnel and effectively improve efficiency, and at the same time, also construct a mapping relationship between requirement information, code repository information, code branch information, code merge request and sonarQube project, that is, construct a mapping relationship based on requirements and static scanning results, and set the required test access indicators based on requirements to ensure the overall code quality of requirements. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 The flowchart of the incremental scanning report generation method based on SonarQube in an embodiment of the present invention.
[0018] Figure 2 It is a schematic diagram of an incremental scanning report generation system based on SonarQube in an embodiment of the present invention.
[0019] Figure 3 Schematic diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION
[0020] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific implementation methods in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0021] Embodiment 1
[0022] Reference Figure 1 , a SonarQube-based incremental scanning report generation method is used for an incremental scanning platform, a code repository hosting platform interacting with the incremental scanning platform, and a SonarQube platform interacting with the incremental scanning platform, the method comprising the following steps.
[0023] S1. The code repository hosting platform obtains the code merge request submitted by the developer and triggers the Webhook mechanism to send the code merge request to the incremental scanning platform.
[0024] In this embodiment, the code merge request includes demand branch information and online branch information, wherein the demand branch information includes multiple code branch information, and the online branch information includes multiple release branch information.
[0025] In this embodiment, it is assumed that there are multiple requirements on the release date. Taking a certain requirement A as an example, the test requires that requirement A releases code scanning access control data with no more than 10 bad access control smells. It should be understood that the "10" here is the number of identifications, and the types include bugs, vulnerabilities, security hotspots, and odors. Specific requirements for each type are set based on scanning rules. Scanning rules can be self-developed, with the help of SonarQube's built-in scanning rules, or imported from certain companies' rule packages (such as Alibaba's open source rule packages); in addition, the development of requirement A involves the modification of three microservices, and the three microservices correspond to three code repositories, namely, code repository user, code repository finance, and code repository risk. In order to realize the change in requirements, the three code repositories pull code branches dev1, code branch dev2, and code branch d respectively. ev3, the release branch is release1, all code branches dev1, code branch dev2, and code branch dev3 need to be merged into the release branch release1 before they can go online. Therefore, after the demand development is completed, the developer submits a code merge request through the code repository hosting platform, where the code merge request includes source demand branch information and target online branch information. The source demand branch information includes multiple code branch information. Here, the source demand branch information includes code branch dev1, code branch dev2, and code branch dev3. The target online branch information includes the release branch release1, that is, three merge requests are submitted in a distributed manner, from the demand branches dev1, dev2, and dev3 to the release branch release1.
[0026] S2. The incremental scanning platform receives the code merge request sent by the code repository hosting platform, and obtains the relationship information among the requirement information, code repository information, and code branch information entered by the developer. It creates the corresponding sonarQube project on the SonarQube platform through the API interface and configures the corresponding scanning rule information. At the same time, it builds a mapping relationship between the requirement information, code repository information, code branch information, code merge request and sonarQube project, and sends the mapping relationship to the code repository hosting platform.
[0027] In this embodiment, the developer enters the relationship information among the requirement information, code repository information, and code branch information on the incremental scanning platform, as shown in Table 1 below.
[0028] Table 1 Relationship between requirement information, code repository information, and code branch information
[0029] Request Information Code repository information Code branch information A user dev1 A finance dev2 A risk dev3
[0030] In this embodiment, a corresponding sonarQube project is created on the SonarQube platform through an API interface and the corresponding scanning rule information is configured. At the same time, a mapping relationship between requirement information, code repository information, code branch information, code merge requests and sonarQube projects is constructed on the incremental scanning platform. The specific mapping relationship is shown in Table 2 below.
[0031] Table 2 Mapping relationship
[0032]
[0033] S3. The incremental scanning platform performs code trimming on the changed files in the local code repository according to the code merge request, generates files for static analysis, and executes the sonarScanner of the incremental scanning platform. The sonarScanner pulls the sonarQube project in the SonarQube platform and triggers the scan, generates the scan information, and sends the scan information to the SonarQube platform.
[0034] In this embodiment, the incremental scanning platform performs code tailoring on the changed files in the local code repository according to the code merge request to generate files for static analysis, specifically including: Step 1, repository pull, using the JGIT open source library to clone the code repository on the remote code repository hosting platform to the local code repository, and at the same time, switch the local code repository to the code branch;
[0035] Step 2: Get the path of the changed file, call the code repository hosting platform, compare the APIs of the difference between the demand branch and the online branch, get the path of the changed file, the changed file and its modification type, and generate the corresponding task to be trimmed; Step 3: Dynamic resource allocation: Get the task to be trimmed based on the thread pool queue quantity strategy and the file change quantity strategy, and dynamically match the corresponding thread pool; Step 4: Initial trimming: Get the task to be trimmed in the thread pool, and based on the modification type contained in the task to be trimmed, perform initial trimming on the changed file contained in the task to be trimmed, retain the changed files with the modification type of new and / or modified, and obtain the initial trimming file; Step 5: Secondary trimming: Call the demand branch information and the online branch information through the code repository hosting platform, use the Myers difference algorithm to calculate the modification parameter data that needs to be modified in the initial trimming file, and perform secondary trimming on the initial trimming file based on the modification parameter data to generate a file for static analysis.
[0036] In this embodiment, the modification parameter data includes newly added code line number range information, deleted code line number range information, and modified code line number range information.
[0037] In this embodiment, in step one, the repository is pulled, and the code repository on the remote code repository hosting platform is cloned to the local code repository using the third-party package JGIT. At the same time, the local code repository is switched to the code branch. For example, the third-party package JGIT is used to clone the code repository user, the code repository finance, and the code repository risk on the remote code repository hosting platform to the local code repository. At the same time, the local code repository is switched to the code branch dev1, the code branch dev2, and the code branch dev3.
[0038] In this embodiment, in step 2, the changed file path is obtained, and the API for comparing branch differences of the code repository hosting platform is called. The comparison objects are the source requirement branch information and the target online branch information in the code merge request. The changed file path, the changed file and its modification type are obtained, where the modification type includes the new code type, the deleted code type and the modified code type, and corresponding tasks to be trimmed are generated.
[0039] In this embodiment, in step three, dynamic resource allocation is performed to match the fast and slow thread pools according to the number of file change ranges to be modified, so as to prevent the analysis of small demands blocked by demands with many changes. There are two main indicators that affect the allocation, namely the number of fast and slow thread pool queues and the number of file changes. Based on the thread pool queue number strategy and the file change number strategy, the task to be trimmed is obtained, and the corresponding thread pool is dynamically matched. For example, the preset file change number threshold is 1000, and the thread pool queue number threshold is 100. When the file change number of the changed files in the task to be trimmed exceeds 1000, the task to be trimmed is temporarily scheduled to enter the slow thread pool, and then it is determined whether the number of slow thread pool queues exceeds 100. If it exceeds 100, the number of fast and slow thread pool queues is compared, and the thread pool with less queues is matched.
[0040] In this embodiment, in step 4, the first trimming is performed to obtain the tasks to be trimmed in the thread pool, and based on the modification types contained in the tasks to be trimmed, the changed files are divided into newly added files, deleted files and modified files according to the modification types, and the changed files contained in the tasks to be trimmed are trimmed for the first time, only the newly added files and modified files are retained, and the unchanged files and deleted files are trimmed to obtain the first trimmed files.
[0041] In this embodiment, in step 4, secondary trimming is performed, and the detailed contents of the files corresponding to the demand branch and the online branch are called through the code warehouse hosting platform, and the Myers difference algorithm is used to calculate the modification parameter data that needs to be modified in the initial trimming file, that is, the range information of the number of new code lines, the range information of the number of deleted code lines, and the range information of the number of modified code lines that need to be modified are calculated, specifically including: for the code lines that are not involved in the modification, the text is replaced with a blank line; based on performance considerations, for the code line range of the number of new code lines and the number of modified code lines that do not reach the preset modification amount, the text is also replaced with a blank line. Lines, and finally generate a file that can be used for static analysis. For example, modify the changed file a.java, which has a total of 1000 lines of code. The Myers difference algorithm is used to calculate the modification parameter data, and the number of newly added code lines that need to be modified is in the range of [1 to 10], the number of deleted code lines is in the range of [200 to 300], and the number of modified code lines is in the range of [100 to 110]. The remaining code lines are unchanged. The primary trimming file is subjected to secondary trimming processing. After the secondary trimming processing, only the incremental code lines of [1 to 10] and [100 to 110] are retained, and the remaining lines are set to blank lines to generate a file that can be used for static analysis.
[0042] In this embodiment, sonarScanner pulls the sonarQube project in the SonarQube platform and triggers a scan to generate scan information, specifically including: sonarScanner pulls the sonarQube project in the SonarQube platform and triggers a scan, scans the incremental code based on the scan rule information corresponding to the sonarQube project and the incremental report indicator information preset in the incremental code scan report, and during the scanning process, polls to obtain the code scanning access control data contained in the scan rule information, and generates scan information containing multiple incremental report indicator data.
[0043] In this embodiment, sonarScanner is a command line interface tool of the SonarQube platform, which is used to scan local sonarQube projects and send the results to the SonarQube platform for analysis; wherein, sonarScanner is installed in the incremental scanning platform in advance, and the incremental scanning platform executes sonarScanner. sonarScanner is responsible for pulling the sonarQube project in the SonarQube platform and triggering the scan, generating scan information, and sending the scan information to the SonarQube platform after the scan is completed.
[0044] In this embodiment, the sonnarScanner command is executed to perform scanning. Through the diversion of the fast and slow thread pools and two trimming operations, the scanning scope can be effectively reduced, thereby improving the scanning efficiency. At the same time, the file content used for static analysis only includes modified code (i.e., incremental code). Therefore, the scanning report generated by the SonarQube platform is an incremental code scanning report.
[0045] S4. The SonarQube platform receives the scanning information and generates an incremental code scanning report based on the scanning information.
[0046] In this embodiment, the SonarQube platform receives scanning information and generates an incremental code scanning report based on the scanning information, specifically including: the SonarQube platform receives scanning information, based on the incremental report indicator information preset in the incremental code scanning report, sums up multiple incremental report indicator data to obtain comprehensive incremental report indicator data, compares the comprehensive incremental report indicator data with the code scanning access control data, and determines whether the comprehensive incremental report indicator data is less than or equal to the code scanning access control data; if the comprehensive incremental report indicator data is less than or equal to the code scanning access control data, then generates an incremental code scanning report for releasing the corresponding requirement information; if the comprehensive incremental report indicator data is greater than the code scanning access control data, then generates an incremental code scanning report for not releasing the corresponding requirement information.
[0047] In this embodiment, for example, the scanning information is shown in Table 3 below. Requirement A involves three code repositories. After scanning processing, scanning information containing three incremental report indicator data is obtained. The three incremental report indicator data are added together to obtain comprehensive incremental report indicator data (i.e., 8+5+5=18). The comprehensive incremental report indicator data (i.e., 18) is greater than the code scanning access control data (the preset test requires that requirement A must not release more than 10 access control bad smells). Therefore, requirement A does not meet the online requirements, and it is necessary to develop and adjust the code to reduce the number of bad smells, and generate an incremental code scanning report that does not release the corresponding requirement information.
[0048] Table 3 Scanning information
[0049]
[0050] In this embodiment, it also includes: the SonarQube platform is provided with a visual interface for developers to view incremental code scanning reports.
[0051] In this embodiment, developers can view the report details of the incremental code scanning report in the visual interface of the SonarQube platform. At the same time, the SonarQube platform can also generate a corresponding link for each incremental code scanning report and send the link to the incremental scanning platform. At the same time, an overview of the scanning information can be displayed on the incremental scanning platform, such as the number of bugs, bad tastes, vulnerabilities, etc., as well as a link to view the corresponding incremental code scanning report.
[0052] In this embodiment, the code repository hosting platform involved in the present invention may be a GitLab platform, and the SonarQube platform involved is a SonarQube community edition platform.
[0053] Embodiment 2
[0054] This embodiment provides an incremental scanning report generation device based on SonarQube on the basis of the above embodiment 1. Figure 2 , used to implement the steps of the incremental scanning report generation method based on SonarQube described in the above embodiment 1, the device mainly includes: an incremental scanning platform, a code repository hosting platform and a SonarQube platform.
[0055] In this embodiment, the code repository hosting platform includes a code merging request module and a request sending module.
[0056] Furthermore, the code merge request module is used to obtain the code merge requests submitted by developers.
[0057] Furthermore, a request sending module is used to trigger the Webhook mechanism and send a code merge request to the incremental scanning platform.
[0058] In this embodiment, the incremental scanning platform includes a request receiving module, an information input module, a sonarQube project building module, a mapping relationship building module, a code clipping module and a scanning module.
[0059] Furthermore, the request receiving module is used to receive the code merge request sent by the code repository hosting platform.
[0060] Furthermore, the information entry module is used to obtain the relationship information among the requirement information, code repository information, and code branch information entered by the developer.
[0061] Furthermore, the sonarQube project building module is used to create a corresponding sonarQube project on the SonarQube platform through an API interface and configure the corresponding scanning rule information.
[0062] Furthermore, a mapping relationship building module is used to build a mapping relationship between requirement information, code repository information, code branch information, code merge request and sonarQube project, and send the mapping relationship to the code repository hosting platform.
[0063] Furthermore, the code trimming module is used to perform code trimming on the changed files in the local code repository according to the code merge request, and generate files for static analysis.
[0064] Furthermore, the scanning module is used to execute the sonarScanner of the incremental scanning platform. The sonarScanner pulls the sonarQube project in the SonarQube platform and triggers the scan, generates the scan information, and sends the scan information to the SonarQube platform.
[0065] In this embodiment, the SonarQube platform includes an information receiving module and a report generating module.
[0066] Furthermore, the information receiving module is used to receive scanning information.
[0067] Furthermore, the report generation module is used to generate an incremental code scanning report based on the scanning information.
[0068] Embodiment 3
[0069] This embodiment further provides an electronic device based on the above embodiment 1. Figure 3 , Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0070] like Figure 3 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In RAM 303, various programs and data required for the operation of the electronic device are also stored. The processing device 301, ROM 302, and RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.
[0071] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, etc., an output device 307 including, for example, a liquid crystal display (LCD), a speaker, etc., a storage device 308 including, for example, a magnetic tape, a hard disk, etc., and a communication device 309. The communication device 309 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Figure 3 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.
[0072] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.
[0073] Embodiment 4
[0074] Based on the above-mentioned embodiment 1, this embodiment further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned method are implemented.
[0075] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0076] In this embodiment, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperTextTransferProtocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0077] The computer-readable medium may be included in the device, or may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains training data, converts the training data into initial data; determines an initial rule base based on the initial data, and optimizes the parameters of the initial rule base to obtain a target rule base; calculates the rules in the target rule base according to a preset activation weight calculation formula to obtain activation weights; determines abnormal information according to the test data and the activation weights.
[0078] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0079] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0080] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The units described may also be provided in a processor, for example, may be described as: a processor including a data acquisition unit, a rule determination unit, a weight calculation unit, and an anomaly determination unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the data acquisition unit may also be described as a "unit for acquiring training data".
[0081] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc.
[0082] Obviously, it should be understood by those skilled in the art that the above-mentioned various steps of the present invention can be performed in a manner different from the present invention, and the simulation method and experimental equipment include but are not limited to the above description. The above-mentioned various steps of the present invention can be performed in an order different from that here in some cases, and the steps shown or described above can be performed separately. Therefore, the present invention is not limited to any specific combination of hardware and software.
[0083] The above contents are further detailed descriptions of the present invention in combination with specific implementation methods, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the scope of protection of the present invention.
Claims
1. The incremental scanning report generation method based on SonarQube is characterized by: For an incremental scanning platform, a code repository hosting platform interacting with the incremental scanning platform, and a SonarQube platform interacting with the incremental scanning platform, the method comprising: The code repository hosting platform obtains the code merge request submitted by the developer and triggers the Webhook mechanism to send the code merge request to the incremental scanning platform; The incremental scanning platform receives the code merge request sent by the code repository hosting platform, and obtains the relationship information among the requirement information, code repository information, and code branch information entered by the developer. It creates the corresponding sonarQube project on the SonarQube platform through the API interface and configures the corresponding scanning rule information. At the same time, it builds the mapping relationship between the requirement information, code repository information, code branch information, code merge request, and sonarQube project, and sends the mapping relationship to the code repository hosting platform; The incremental scanning platform performs code trimming on the changed files in the local code repository according to the code merge request, generates files for static analysis, and executes the sonarScanner of the incremental scanning platform. The sonarScanner pulls the sonarQube project in the SonarQube platform and triggers scanning, generates scanning information, and sends the scanning information to the SonarQube platform. The SonarQube platform receives the scan information and generates an incremental code scan report based on the scan information.
2. The incremental scanning report generation method based on SonarQube according to claim 1 is characterized in that: The code merge request includes demand branch information and online branch information, wherein the demand branch information includes multiple code branch information, and the online branch information includes multiple release branch information.
3. The incremental scanning report generation method based on SonarQube according to claim 2 is characterized in that: The incremental scanning platform performs code tailoring on the changed files in the local code repository according to the code merge request, and generates files for static analysis, specifically including: Step 1: Pull the repository and use the JGIT open source library to clone the code repository on the remote code repository hosting platform to the local code repository. At the same time, switch the local code repository to the code branch. Step 2: Get the changed file path, call the code repository hosting platform, compare the APIs of the difference between the demand branch and the online branch, get the changed file path, changed file and its modification type, and generate the corresponding tasks to be trimmed; Step 3: Dynamic resource allocation: based on the thread pool queue quantity strategy and the file change quantity strategy, obtain the tasks to be trimmed and dynamically match the corresponding thread pool; Step 4: Initial trimming: obtaining the tasks to be trimmed in the thread pool, and performing initial trimming processing on the changed files contained in the tasks to be trimmed based on the modification types contained in the tasks to be trimmed, retaining the changed files whose modification types are newly added and / or modified, and obtaining the initial trimming files; Step 5: Secondary trimming: Call the required branch information and online branch information through the code repository hosting platform, use the Myers differential algorithm to calculate the modified parameter data that needs to be modified in the initial trimming file, and perform secondary trimming on the initial trimming file based on the modified parameter data to generate a file for static analysis.
4. The incremental scanning report generation method based on SonarQube according to claim 3 is characterized in that: The modification parameter data includes newly added code line number range information, deleted code line number range information and modified code line number range information.
5. The incremental scanning report generation method based on SonarQube according to claim 3 is characterized in that: The sonarScanner pulls the sonarQube project in the SonarQube platform and triggers scanning to generate scanning information, specifically including: sonarScanner pulls the sonarQube project in the SonarQube platform and triggers scanning, scans the incremental code based on the scanning rule information corresponding to the sonarQube project and the incremental report indicator information preset in the incremental code scanning report, and during the scanning process, polls to obtain the code scanning access control data contained in the scanning rule information, and generates scanning information containing multiple incremental report indicator data.
6. The incremental scanning report generation method based on SonarQube according to claim 5 is characterized in that: The SonarQube platform receives scanning information and generates an incremental code scanning report based on the scanning information, specifically including: the SonarQube platform receives scanning information, based on the incremental report indicator information preset in the incremental code scanning report, sums multiple incremental report indicator data to obtain comprehensive incremental report indicator data, compares the comprehensive incremental report indicator data with the code scanning access control data, and determines whether the comprehensive incremental report indicator data is less than or equal to the code scanning access control data. If the comprehensive incremental report indicator data is less than or equal to the code scanning access control data, an incremental code scanning report is generated for releasing the corresponding requirement information; if the comprehensive incremental report indicator data is greater than the code scanning access control data, an incremental code scanning report is generated for not releasing the corresponding requirement information.
7. The incremental scanning report generation method based on SonarQube according to claim 1 is characterized in that: Also includes: The SonarQube platform provides a visual interface for developers to view incremental code scanning reports.
8. An incremental scanning report generation device based on SonarQube, used for an incremental scanning platform, a code repository hosting platform interacting with the incremental scanning platform, and a SonarQube platform interacting with the incremental scanning platform, characterized in that: include: The code repository hosting platform includes a code merge request module and a request sending module; wherein the code merge request module is used to obtain the code merge request submitted by the developer; the request sending module is used to trigger the Webhook mechanism to send the code merge request to the incremental scanning platform; The incremental scanning platform includes a request receiving module, an information entry module, a sonarQube project building module, a mapping relationship building module, a code clipping module and a scanning module; wherein the request receiving module is used to receive a code merge request sent by a code repository hosting platform; the information entry module is used to obtain the relationship information among the requirement information, code repository information and code branch information entered by the developer; the sonarQube project building module is used to create a corresponding sonarQube project on the SonarQube platform through an API interface and configure the corresponding scanning rule information; the mapping relationship building module is used to build a mapping relationship between requirement information, code repository information, code branch information, code merge request and sonarQube project, and send the mapping relationship to the code repository hosting platform; the code clipping module is used to perform code clipping on the changed files in the local code repository according to the code merge request to generate files for static analysis; the scanning module is used to execute the sonarScanner of the incremental scanning platform, sonarScanner pulls the sonarQube project in the SonarQube platform and triggers scanning, generates scanning information, and sends the scanning information to the SonarQube platform; The SonarQube platform includes an information receiving module and a report generating module; wherein the information receiving module is used to receive scanning information; and the report generating module is used to generate an incremental code scanning report based on the scanning information.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
