Software development test method and system, electronic equipment and storage medium

Abstract

The invention provides a software development testing method and system, electronic equipment and a storage medium, and the method comprises the steps: building connection with a testing environment in response to a performance testing instruction which is used for triggering a performance test of a program code of to-be-tested software and is generated based on a compilation success identifier of the program code; deploying an installation package generated by compiling the program code into a test environment, and performing performance test to obtain a target test value of a performance index; and if the target test value meets a threshold condition, determining that the performance test result is passed, thereby completing the software development test, the threshold condition comprising that the increment of the target test value is less than or equal to a preset increment threshold and the target test value is less than or equal to a preset upper limit threshold. After the program code compilation verification is passed, the performance test of the program code is triggered, so that the performance test is completed before the code is put in storage, and the problem that the program code with the performance problem is merged into a development branch warehouse can be effectively prevented.

Description

Technical Field

[0001] The present application relates to the field of software development technology, and in particular to a software development and testing method, system, electronic device, and storage medium. Background Art

[0002] Ensuring software quality and performance is crucial during software development. It's not only key to achieving product functionality requirements, but also crucial for ensuring user experience. To effectively improve software quality and performance, a testing mechanism is introduced into the development process to inspect, verify, and test software code to ensure quality and performance.

[0003] In related technologies, although the verification mechanism in the development process has achieved certain results in ensuring software quality and functionality, there is still the problem of code with performance issues being introduced into the development branch repository. Once the code with performance issues enters the development branch repository, it will have an adverse impact on the performance of the entire system or application, increasing the cost and complexity of subsequent repairs. Summary of the Invention

[0004] In view of the above shortcomings, the present application discloses a software development and testing method, system, electronic device and storage medium, which are used to solve the technical problem of incorporating code with performance issues into the development branch repository during software development and testing, thereby affecting system and application performance.

[0005] In the first aspect, the present application provides a software development testing method, the method comprising: establishing a connection with a test environment in response to a received performance test instruction, the performance test instruction being used to trigger a performance test of the program code of the software to be tested, and generating an identifier based on a successful compilation of the program code; deploying the installation package generated by compiling the program code into the test environment, performing a performance test, and obtaining a target test value of the performance indicator of the program code; if the target test value meets the threshold condition of the performance indicator, determining that the performance test result is a test pass, thereby completing the software development test, the threshold condition including that the increment of the target test value is less than or equal to a preset increment threshold and that the target test value is less than or equal to a preset upper limit threshold.

[0006] In one embodiment of the present application, the method for obtaining the target test value includes: periodically collecting multiple test values of performance indicators in each test point, the test environment includes multiple test points, the performance indicators include CPU occupancy and memory occupancy, and the test values include CPU occupancy test value and memory occupancy test value; for each test point, determining the single-point test value of the CPU occupancy based on the multiple CPU occupancy test values, and determining the single-point test value of the memory occupancy based on the multiple memory occupancy test values; determining the target test value of the CPU occupancy based on the multiple single-point test values of the CPU occupancy, and determining the target test value of the memory occupancy based on the multiple single-point test values of the memory occupancy.

[0007] In one embodiment of the present application, the method for determining the performance test result includes: obtaining the historical test value of the software to be tested, the historical test value being the test value in which the performance indicator meets the threshold condition in the last performance test; calculating the difference between the target test value and the historical test value to obtain the increment; if the increment is less than or equal to the increment threshold, and the target test value is less than or equal to the upper limit threshold, determining that the performance test result is a passed test; if the increment is greater than the increment threshold, and / or the target test value is greater than the upper limit threshold, determining that the performance test result is a failed test.

[0008] In one embodiment of the present application, the method for determining the performance test result also includes: determining the code type of the program code based on the code type identification field in the program code, the code type including vulnerability repair type and function extension type; if the program code is the vulnerability repair type, determining the performance test result based on the first incremental threshold, the upper limit threshold, the target test value and the historical test value, the incremental threshold including the first incremental threshold; if the program code is the function extension type, determining the performance test result based on the second incremental threshold, the upper limit threshold, the target test value and the historical test value, the incremental threshold also including the second incremental threshold.

[0009] In one embodiment of the present application, after the performance test result is determined, it also includes: feeding back the performance test result to each test point in the test environment, and if the performance test result is that the test fails, performing error troubleshooting on each test point; generating an email based on the performance test result and sending it to a target email address, and if the performance test result is that the test passes, conducting a warehousing review of the program code, and if the performance test result is that the test fails, terminating the warehousing operation of the program code and performing error troubleshooting on the program code.

[0010] In one embodiment of the present application, after the software development test is completed, it also includes: sending a notification of test passing and merging the program code into the development branch repository; using the target test value to update the historical test value of the software to be tested as a basis for calculating the incremental test value during the next performance test of the software to be tested.

[0011] In one embodiment of the present application, before responding to the received performance test instruction, it also includes: responding to the software development test instruction, performing a specification check on the program code; if the specification check fails, terminating the software development test; if the specification check passes, sending a notification of passing the specification check, and performing a static code check on the program code; if the static code check fails, terminating the software development test; if the static code check passes, sending a notification of passing the static code check, and performing compilation verification of the program code; if the compilation verification fails, terminating the software development test; if the compilation verification passes, sending a notification of passing the compilation verification, and performing performance testing of the program code.

[0012] In the second aspect, the present application provides a software development testing system, which includes: a connection management module for establishing a connection with a test environment in response to a received performance testing instruction, wherein the performance testing instruction is used to trigger a performance test of the program code of the software to be tested, and is generated based on a successful compilation identifier of the program code; a performance testing module for deploying the installation package generated by compiling the program code into the test environment, performing a performance test, and obtaining a target test value of the performance indicator of the program code; a performance analysis module for determining that the performance test result is a pass if the target test value meets the threshold condition of the performance indicator, thereby completing the software development test, wherein the threshold condition includes that the increment of the target test value is less than or equal to a preset increment threshold and that the target test value is less than or equal to a preset upper limit threshold.

[0013] In a third aspect, the present application provides an electronic device comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the electronic device implements the software development and testing method as described in the first aspect.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor of a computer, the computer is caused to execute the software development and testing method as described in the first aspect.

[0015] As described above, the software development and testing method, system, electronic device, and storage medium provided by the embodiments of the present application have the following beneficial effects:

[0016] First, in response to the received performance test instruction, a connection is established with the test environment. The performance test instruction is used to trigger the performance test of the program code of the software to be tested, and is generated based on the compilation success identifier of the program code. Then, the installation package generated by the program code compilation is deployed to the test environment for performance testing to obtain the target test value of the performance indicator of the program code. If the target test value meets the threshold condition of the performance indicator, the performance test result is determined to be a pass, thereby completing the software development test, wherein the threshold condition includes that the increment of the target test value is less than or equal to the preset increment threshold and the target test value is less than or equal to the preset upper limit threshold. By responding to the performance test instruction generated based on the compilation success identifier of the program code, the performance test of the program code is performed, that is, the performance test of the program code is triggered after the program code compilation verification is passed, thereby realizing automated performance testing before the program code is put into the warehouse, which can effectively prevent the problem of merging program code with performance problems into the development branch warehouse, thereby preventing program code with non-standard performance from being merged into the software version, improving the quality and reliability of software development, and ensuring the performance of the system and application.

[0017] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are incorporated into and constitute a part of the specification, illustrating embodiments consistent with the present application and, together with the specification, serving to explain the principles of the present application. It is obvious that the drawings described below are merely some embodiments of the present application, and a person of ordinary skill in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0019] Figure 1 This is a schematic diagram of an implementation environment of a software development and testing system shown in an exemplary embodiment of the present application;

[0020] Figure 2 is a flowchart of a software development and testing method shown in an exemplary embodiment of the present application;

[0021] Figure 3 is a flow chart illustrating a method of determining a performance test result according to an exemplary embodiment of the present application;

[0022] Figure 4 This is a flowchart of a specific software development and testing method shown in an exemplary embodiment of the present application;

[0023] Figure 5 is a flowchart of another software development and testing method shown in an exemplary embodiment of the present application;

[0024] Figure 6 is a block diagram of a software development and testing system shown in an exemplary embodiment of the present application;

[0025] Figure 7 This is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will describe the embodiments of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand the other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for the purpose of illustrating the present application and are not intended to limit the scope of protection of the present application.

[0027] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. Therefore, the illustrations only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the shape, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0028] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0029] Regarding the verification mechanism in software development, usually after a software development engineer develops a bug fix code or a function extension code, he submits the code to the code repository, which first triggers a specification check, then a static code check, and then a compilation verification. The code is then merged into the development branch repository after multiple reviews by the application engineer. After being merged into the development branch repository, a software version is compiled, and the test engineer performs performance testing on the version. However, the inventors of this application have found that the performance testing of software versions still has problems of lag and insufficient automation. Lagging testing may cause code with performance issues to be introduced into the development branch repository. Once the code with performance issues enters the development branch repository, it will have an adverse effect on the performance of the entire system, increasing the cost and complexity of subsequent repairs. Performance testing is often done by test engineers after the code is stored in the repository, which has manual dependencies and affects the efficiency and accuracy of performance testing.

[0030] Therefore, see Figure 1 , Figure 1 FIG. 1 is a schematic diagram of an implementation environment of a software development and testing system according to an exemplary embodiment of the present application. Figure 1 As shown, the implementation environment may include a software development and testing system 110 and a computer device 120. The software development and testing system 110 may be set in the computer device 120 for performing software development and testing. The computer device 120 may be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, a neural network computer, and the like. Through the software development and testing system 120, in response to the performance test instructions generated based on the successful compilation mark of the program code, the performance test of the program code is performed, that is, the performance test of the program code is triggered after the program code compilation verification is passed, thereby realizing automated performance testing before the program code is put into storage, and being able to effectively prevent the problem of merging program code with performance problems into the development branch warehouse, thereby preventing program code with non-standard performance from being merged into the software version, improving the quality and reliability of software development, and ensuring the performance of the system and application.

[0031] See Figure 2 , Figure 2 This is a flowchart of a software development and testing method shown in an exemplary embodiment of the present application. This method can be applied to Figure 1 The implementation environment shown is specifically implemented by the software development and testing system in the implementation environment. It should be understood that the method can also be applied to other exemplary implementation environments and be specifically implemented by devices in other implementation environments. This embodiment does not limit the implementation environment to which the method is applicable.

[0032] like Figure 2 As shown, in an exemplary embodiment, the software development testing method includes at least steps S210 to S230, which are described in detail as follows:

[0033] Step S210 , establishing a connection with a test environment in response to a received performance test instruction, wherein the performance test instruction is used to trigger a performance test of the program code of the software to be tested, and a compilation success flag is generated based on the program code.

[0034] Step S220 , deploying the installation package generated by compiling the program code into a test environment, performing performance testing, and obtaining target test values of performance indicators of the program code.

[0035] Step S230: If the target test value meets the threshold conditions of the performance indicator, the performance test result is determined to be a passed test, thereby completing the software development test. The threshold conditions include that the increment of the target test value is less than or equal to the preset increment threshold and the target test value is less than or equal to the preset upper limit threshold.

[0036] Among them, the test environment is an independent platform or system designed and configured to test the program code of the software to be tested. The test environment simulates the conditions required for the actual operation of the software to conduct comprehensive and effective testing of its program code.

[0037] In step S210, a performance test instruction is generated based on the compilation success flag of the program code of the software to be tested to trigger the performance test of the program code. In other words, the performance test is started after the program code is successfully compiled. In addition, before executing the performance test, it is necessary to first connect to the test environment used for performance testing.

[0038] In step S220, after ensuring a reliable connection with the test environment, the installation package generated by compiling the program code is deployed to the test environment, and performance testing is performed to obtain the target test values of the performance indicators of the program code, thereby monitoring the performance indicator data related to the program running status.

[0039] In step S230, the target test value is judged whether it meets the standard through the threshold condition. If the increment of the target test value does not exceed the preset increment threshold and the target test value does not exceed the preset upper limit threshold, the result of the performance test can be determined to be a passed test, thereby completing the test of this round of software development, indicating that the program code can be merged into the development branch repository.

[0040] In this embodiment, performance testing of the program code is performed in response to a performance testing instruction generated based on a successful compilation flag of the program code. That is, performance testing of the program code is triggered after the program code compilation verification passes. This implements automated performance testing before the program code is stored. This effectively prevents program code with performance issues from being incorporated into the development branch repository, thereby preventing program code with substandard performance from being incorporated into the software version. This improves the quality and reliability of software development and ensures the performance of the system and applications. Furthermore, automated performance testing improves the efficiency and accuracy of performance testing.

[0041] For example, the performance indicators include, but are not limited to, CPU occupancy and memory occupancy, and may also include response time, throughput, and error rate.

[0042] In one embodiment, a method for obtaining a target test value includes: periodically collecting multiple test values of performance indicators in each test point, the test environment includes multiple test points, the performance indicators include CPU occupancy and memory occupancy, and the test values include CPU occupancy test values and memory occupancy test values; for each test point, determining a single-point test value of CPU occupancy based on multiple CPU occupancy test values, and determining a single-point test value of memory occupancy based on multiple memory occupancy test values; determining a target test value of CPU occupancy based on multiple single-point test values of CPU occupancy, and determining a target test value of memory occupancy based on multiple single-point test values of memory occupancy.

[0043] In this embodiment, taking into account the problem that the number of test samples of performance indicators is limited and cannot fully reflect the actual running status of the program code, and thus cannot accurately analyze the program code performance of the software to be tested, multiple test points are configured in the test environment. By collecting multiple data from multiple test points, the final target test value is determined. In this way, the number of sample data is guaranteed, the one-sidedness of single data can be effectively avoided, and abnormal values can be effectively filtered, making the target test value more objective and accurate.

[0044] Exemplarily, periodically collecting multiple test values of the performance indicator at each test point includes: collecting multiple test values of the performance indicator at each test point based on preset time intervals within a preset test duration.

[0045] For example, if the test duration is 30 minutes, the time interval between each collection is 30 seconds.

[0046] Exemplarily, for each test point, a single point test value of the CPU occupancy is determined based on multiple CPU occupancy test values. The average of the multiple CPU occupancy test values can be calculated as the single point test value of the CPU occupancy, or the median, mode, etc. of the multiple CPU occupancy test values can be determined as the single point test value of the CPU occupancy.

[0047] Exemplarily, for each test point, a single point test value of memory occupancy is determined based on multiple memory occupancy test values. The average of the multiple memory occupancy test values can be calculated as the single point test value of memory occupancy, or the median, mode, etc. of the multiple memory occupancy test values can be determined as the single point test value of memory occupancy.

[0048] Exemplarily, the target test value of the CPU occupancy rate is determined based on multiple single-point test values of the CPU occupancy rate. The average value of the multiple single-point test values of the CPU occupancy rate can be calculated as the target test value of the CPU occupancy rate, or the median, mode, etc. of the multiple single-point test values of the CPU occupancy rate can be determined as the target test value of the CPU occupancy rate.

[0049] Exemplarily, the target test value of memory occupancy is determined based on multiple single-point test values of memory occupancy. The average value of multiple single-point test values of memory occupancy can be calculated as the target test value of memory occupancy, or the median, mode, etc. of multiple single-point test values of memory occupancy can be determined as the target test value of memory occupancy.

[0050] In one embodiment, a method for determining a performance test result includes: obtaining historical test values of the software to be tested, where the historical test values are test values in which the performance indicators in the last performance test meet the threshold conditions; calculating the difference between the target test value and the historical test value to obtain an increment; if the increment is less than or equal to the increment threshold, and the target test value is less than or equal to the upper limit threshold, determining that the performance test result is a passed test; if the increment is greater than the increment threshold, and / or the target test value is greater than the upper limit threshold, determining that the performance test result is a failed test.

[0051] The last performance test refers to the performance test in the previous version of software development test.

[0052] In this embodiment, whether the target test value meets the performance standard is comprehensively judged from two dimensions: whether the increment of the target test value relative to the historical test value meets the conditions of the enhancement threshold and whether the target test value meets the conditions of the upper limit threshold. In this way, the reliability of the performance test results is ensured.

[0053] Exemplarily, the performance indicators include CPU occupancy and memory occupancy, the target test values include the target test value of CPU occupancy and the target test value of memory occupancy, and the method for determining the performance test result is specifically as follows: if the increment corresponding to the target test value of CPU occupancy is less than or equal to the corresponding increment threshold, the increment corresponding to the target test value of memory occupancy is less than or equal to the corresponding increment threshold, and the target test value of CPU occupancy is less than or equal to the corresponding upper limit threshold, and the target test value of memory occupancy is less than or equal to the corresponding upper limit threshold, then the performance test result is determined to be a passed test; if there is at least one of the following situations: the increment corresponding to the target test value of CPU occupancy is greater than the corresponding increment threshold, the increment corresponding to the target test value of memory occupancy is greater than the corresponding increment threshold, the target test value of CPU occupancy is greater than the corresponding upper limit threshold, and the target test value of memory occupancy is greater than the corresponding upper limit threshold, then the performance test result is determined to be a failed test.

[0054] In this exemplary embodiment, the incremental thresholds set corresponding to the two performance indicators of CPU occupancy and memory occupancy may be the same or different, and the upper limit thresholds set corresponding to the two performance indicators of CPU occupancy and memory occupancy may be the same or different, and their specific values are set according to software requirements during the software design phase.

[0055] In one embodiment, the method for determining the performance test results also includes: determining the code type of the program code based on the code type identification field in the program code, the code types including vulnerability repair type and function extension type; if the program code is of vulnerability repair type, determining the performance test results based on the first incremental threshold, the upper limit threshold, the target test value and the historical test value, the incremental threshold including the first incremental threshold; if the program code is of function extension type, determining the performance test results based on the second incremental threshold, the upper limit threshold, the target test value and the historical test value, the incremental threshold also including the second incremental threshold.

[0056] In this embodiment, considering that different types of program codes may have different incremental requirements for performance indicators, when performing performance analysis, the type of program code is first determined, and then the performance is judged based on the specific type according to the corresponding incremental threshold. In this way, the reliability of the performance test results is further guaranteed.

[0057] In addition, for the same software to be tested, whether it is bug fixing or function expansion during software development, the upper limit requirements of the performance indicators of the program code are the same, that is, the upper limit thresholds are the same.

[0058] Exemplarily, if the program code is of the vulnerability repair type, the performance test result is determined based on the first incremental threshold, the upper limit threshold, the target test value and the historical test value, specifically: the difference between the target test value and the historical test value is calculated; if the difference is less than or equal to the first incremental threshold, and the target test value is less than or equal to the upper limit threshold, the performance test result is determined to be a passed test; if the difference is greater than the first incremental threshold, and / or the target test value is greater than the upper limit threshold, the performance test result is determined to be a failed test.

[0059] Exemplarily, if the program code is of a functional extension type, the performance test result is determined based on the second incremental threshold, the upper limit threshold, the target test value and the historical test value, specifically: the difference between the target test value and the historical test value is calculated; if the difference is less than or equal to the second incremental threshold, and the target test value is less than or equal to the upper limit threshold, the performance test result is determined to be a passed test; if the difference is greater than the second incremental threshold, and / or the target test value is greater than the upper limit threshold, the performance test result is determined to be a failed test.

[0060] Exemplarily, the first incremental threshold is smaller than the second incremental threshold. Generally speaking, the incremental performance indicator of the vulnerability repair type program code is smaller than the incremental performance indicator of the function expansion type program code. Therefore, the set first incremental threshold is usually also smaller than the second incremental threshold.

[0061] For example, for vulnerability-fixing program codes, the incremental thresholds corresponding to the two performance indicators of CPU usage and memory usage are 2%, and the corresponding upper limit thresholds are 20%. Therefore, the incremental corresponding to the target test value of CPU usage and the incremental corresponding to the target test value of memory usage need to be ≤2%, and the target test value of CPU usage and the target test value of memory usage need to be ≤20% before the performance test result can be determined as passed; for function-expanding program codes, the incremental thresholds corresponding to the two performance indicators of CPU usage and memory usage are 5%, and the corresponding upper limit threshold is 20%. Therefore, the incremental corresponding to the target test value of CPU usage and the incremental corresponding to the target test value of memory usage need to be ≤5%, and the target test value of CPU usage and the target test value of memory usage need to be ≤20% before the performance test result can be determined as passed.

[0062] See Figure 3 , Figure 3 FIG. 1 is a flow chart showing a method of determining a performance test result according to an exemplary embodiment of the present application. Figure 3 As shown, taking the example that the increment thresholds and upper limit thresholds set for the two performance indicators of CPU usage and memory usage are the same, the step of determining the performance test result includes at least steps S310 to S350, which are detailed as follows:

[0063] Step S310, obtaining the program code of the software to be tested;

[0064] Step S320, determining the code type of the program code according to the code type identification field in the program code;

[0065] Step S330 includes steps S331 and S332; wherein, in step S331, if the code type is vulnerability repair type, the incremental threshold corresponding to the CPU occupancy rate and the memory occupancy rate is determined to be the first incremental threshold; and in step S332, if the code type is function expansion type, the incremental threshold corresponding to the CPU occupancy rate and the memory occupancy rate is determined to be the second incremental threshold;

[0066] Step S340 includes step S341 and step S342; wherein, in step S341, it is determined whether the increments of the target test values corresponding to the CPU occupancy rate and the memory occupancy rate are both ≤ the first increment threshold, and whether the target test values corresponding to the CPU occupancy rate and the memory occupancy rate are both ≤ the upper limit threshold, if so, the process proceeds to step S351, if not, the process proceeds to step S352; step S342, it is determined whether the increments of the target test values corresponding to the CPU occupancy rate and the memory occupancy rate are both ≤ the second increment threshold, and whether the target test values corresponding to the CPU occupancy rate and the memory occupancy rate are both ≤ the upper limit threshold, if so, the process proceeds to step S351, if not, the process proceeds to step S352;

[0067] Step S350 includes step S351 and step S352; wherein, in step S351, it is determined that the performance test result is a test pass; and in step S352, it is determined that the performance test result is a test fail.

[0068] In one embodiment, after determining the performance test results, it also includes: feeding back the performance test results to each test point in the test environment, and if the performance test result is that the test fails, then troubleshooting is performed on each test point; generating an email based on the performance test result and sending it to a target email address, and if the performance test result is that the test passes, then conducting a warehousing review of the program code; and if the performance test result is that the test fails, then terminating the warehousing operation of the program code and performing error troubleshooting on the program code.

[0069] In this embodiment, considering that different automated operations will be performed for different performance test results after the performance test, the performance test results will be fed back to each test point and sent to the target email address. At each test point, if the performance test result obtained is that the test fails, the configuration of the test point will be checked for errors. If the performance test result is that the test fails, the corresponding developer will be notified by sending an email to check the program code for errors. Problems can be discovered and fixed in time before the code is put into storage, which can effectively reduce the cost and complexity of repair. If the performance test result is that the test passes, the next step of storage review can be triggered in time.

[0070] In this way, through the comprehensive feedback mechanism of performance test results, the problem can be accurately located before the code is put into the database, and it can be specifically checked whether it is a problem with the program code itself or a problem with the test point configuration. In addition, if the test passes, a quick response can be made to conduct a database review of the program code, ensuring the efficiency and reliability of software development testing.

[0071] In one embodiment, after completing the software development test, it also includes: sending a notification of test passing and merging the program code into the development branch repository; using the target test value to update the historical test value of the software to be tested, as a benchmark for calculating the incremental test value during the next performance test of the software to be tested.

[0072] Among them, the next performance test refers to the performance test in the next version of software development test.

[0073] In this embodiment, after the performance test passes, a notification of test passing is sent to perform subsequent program code warehousing operations. This can effectively prevent program codes with performance issues from being incorporated into the development code library, and historical test values are updated using target test values that meet the standards, providing reliable comparison data for subsequent software development tests of the software to be tested, thereby ensuring the reliability of subsequent performance test results.

[0074] In one embodiment, before responding to the received performance test instruction, it also includes: responding to the software development test instruction, performing a specification check on the program code; if the specification check fails, terminating the software development test; if the specification check passes, sending a notification of passing the specification check, and performing a static code check of the program code; if the static code check fails, terminating the software development test; if the static code check passes, sending a notification of passing the static code check, and performing compilation verification of the program code; if the compilation verification fails, terminating the software development test; if the compilation verification passes, sending a notification of passing the compilation verification, and performing performance testing of the program code.

[0075] In this embodiment, the specification check, static code check and compilation verification before the performance test have a sequential dependency, that is, the next step of the test will not be carried out until the test result of the previous step is passed, and if the test result of any step is failed, the software development test process will be directly terminated, thus avoiding waste of resources.

[0076] See Figure 4 , Figure 4 This is a flowchart of a specific software development and testing method shown in an exemplary embodiment of the present application. Figure 4 As shown, taking in-vehicle software development as an example, the software development environment is an in-vehicle software development environment, the code repository can be a Gerrit (a code review tool based on a distributed version control system) code repository, the server can be a Jenkins (an automated build tool) server, and the test environment is an in-vehicle development board. The steps of software development and testing are detailed as follows:

[0077] First, software development engineers write vulnerability-fixing or function-expanding code in the vehicle software development environment. After writing, they submit the code from the vehicle software development environment to the code repository, which triggers the server to perform software development testing through software development test instructions.

[0078] Next, the server establishes a connection with the test environment, which has 10 test points, namely 10 vehicle-mounted development boards, and deploys the installation package generated by compiling the program code to each vehicle-mounted development board in the test environment;

[0079] Then, various inspection and test instructions are executed on the server. First, the program code is inspected for specifications, and the inspection results are fed back to the code repository in the form of +1 (passed) or -1 (failed). The code repository updates the inspection results. If the inspection result is +1, the static code inspection instruction is used to trigger the server to perform a static code inspection, and the static code inspection results are fed back to the code repository in the form of +1 (passed) or -1 (failed). The code repository updates the static code inspection results. If the static code inspection result is +1, the compilation verification instruction is used to trigger the server to perform coding verification. The coding verification result is fed back, that is, returned to the code repository in the form of +1 (passed) or -1 (failed). The code repository updates the coding verification result. If the coding verification result is +1, the server is triggered to perform performance testing through the performance test instruction, and the performance result is fed back, that is, returned to the code repository in the form of +1 (passed) or -1 (failed), and fed back to each test point and notified to the corresponding developer by email. The code repository updates the performance test result. If the performance test result is +1, it triggers the warehousing review, and when the warehousing review result is +2, the program code is merged into the branch development repository.

[0080] Please continue to see Figure 4 A performance test result item has been added to the code repository. If the performance test result returned by the server is +1, it means that the program code submitted this time meets the performance requirements and can be merged into the branch development repository after passing the review. If the performance test result returned by the server is -1, it means that the program code submitted this time does not meet the performance requirements, and the code entry process will be terminated. A performance testing module has been added to the server, which can connect to the test environment, deploy the installation package generated by the program code compilation to the test environment, perform performance testing, and feedback the performance test results to the test environment, notify the corresponding developer by email, and feedback to the code repository.

[0081] In addition, after the program code is submitted to the code repository, it is only stored in a temporary repository and is not actually merged into the development branch repository. It needs to go through Commit message (i.e. specification check), Sonarqube (i.e. static code check), Verified (coding verification) and performance test in sequence, and the results must all be +1 before it enters the warehousing process for this program code.

[0082] See Figure 5 , Figure 5 FIG. 1 is a flowchart of another software development and testing method shown in an exemplary embodiment of the present application. Figure 5 As shown, the steps of the other software development and testing method include at least steps S510 to S580, which are described in detail as follows:

[0083] Step S510, establishing a connection with a test environment in response to a performance test instruction;

[0084] Step S520: deploying the installation package generated by compiling the program code into a test environment for performance testing;

[0085] Step S530, periodically collecting the CPU occupancy rate and memory occupancy rate of each test point;

[0086] Step S540, calculating an average of a plurality of CPU occupancy rates at each test point, and calculating an average of a plurality of memory occupancy rates at each test point;

[0087] Step S550, calculating the average value of the CPU occupancy rate of all test points, and calculating the average value of the memory occupancy rate of all test points;

[0088] Step S560, determining a performance test result based on the average value of the CPU usage, the average value of the CPU usage in historical tests, the incremental threshold, and the upper threshold, and based on the average value of the memory usage, the average value of the memory usage in historical tests, the incremental threshold, and the upper threshold;

[0089] Step S570: Feedback the performance test results to the test environment, notify the corresponding developer by email, and feed back to the code repository;

[0090] In step S580, if the program code is merged into the development branch repository, the average values of the CPU usage and memory usage calculated this time are used to update the average values of the CPU usage and memory usage in the historical test, so as to serve as a basis for calculating the increment in the next performance test.

[0091] The above-mentioned software development testing method first establishes a connection with the test environment in response to the received performance test instruction, and the performance test instruction is used to trigger the performance test of the program code of the software to be tested, and is generated based on the compilation success identifier of the program code. Then, the installation package generated by the program code compilation is deployed to the test environment for performance testing to obtain the target test value of the performance indicator of the program code. If the target test value meets the threshold condition of the performance indicator, the performance test result is determined to be a pass, thereby completing the software development test, wherein the threshold condition includes that the increment of the target test value is less than or equal to the preset increment threshold and the target test value is less than or equal to the preset upper limit threshold. By responding to the performance test instruction generated based on the compilation success identifier of the program code, the performance test of the program code is performed, that is, the performance test of the program code is triggered after the program code compilation verification is passed, thereby realizing automated performance testing before the program code is put into the warehouse, which can effectively prevent the problem of merging program code with performance problems into the development branch warehouse, thereby preventing program code with non-standard performance from being merged into the software version, improving the quality and reliability of software development, and ensuring the performance of the system and application.

[0092] See Figure 6 , Figure 6 This is a block diagram of a software development and testing system shown in an exemplary embodiment of the present application. The system can be applied to Figure 1 The implementation environment shown is shown. It should be understood that the system can also be applied to other exemplary implementation environments, and this embodiment does not limit the implementation environment to which the system is applicable.

[0093] like Figure 6 As shown, in an exemplary embodiment, the software development and testing system 600 includes at least a connection management module 610, a performance testing module 620, and a performance analysis module 630, which are described in detail as follows:

[0094] The connection management module 610 is configured to establish a connection with the test environment in response to a received performance test instruction, wherein the performance test instruction is configured to trigger a performance test of the program code of the software under test and is generated based on a compilation success flag of the program code;

[0095] The performance testing module 620 is used to deploy the installation package generated by compiling the program code into a test environment, perform performance testing, and obtain target test values of performance indicators of the program code;

[0096] The performance analysis module 630 is used to determine that the performance test result is a pass if the target test value meets the threshold conditions of the performance indicator, thereby completing the software development test. The threshold conditions include that the increment of the target test value is less than or equal to the preset increment threshold and the target test value is less than or equal to the preset upper limit threshold.

[0097] It should be noted that the software development and testing system provided in the above embodiment and the software development and testing method provided in the above embodiment belong to the same concept, wherein the contents of the operations performed by each module have been described in detail in the method embodiment and will not be repeated here.

[0098] See Figure 7 , Figure 7 This is a structural diagram of an electronic device provided by an embodiment of the present application. Figure 7 The following is a schematic diagram showing the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application. Figure 7 The computer system 700 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0099] like Figure 7 As shown, computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to the program stored in read-only memory (ROM) 702 or the program loaded from storage portion 708 into random access memory (RAM) 703, such as executing the method in the above embodiment. Various programs and data required for system operation are also stored in RAM 703. CPU 701, ROM 702 and RAM 703 are connected to each other via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0100] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output section 707 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. Removable media 711, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 710 as needed, so that computer programs read therefrom can be installed into the storage section 708 as needed.

[0101] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from a removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the various functions defined in the system of the present application are executed.

[0102] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computer processor, the computer executes the software development and testing method described above. The computer-readable storage medium may be included in the electronic device described in the above embodiments, or may exist independently and not be incorporated into the electronic device.

[0103] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable computer program. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0104] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. Among them, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0105] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. In some cases, the names of these units do not constitute limitations on the units themselves.

[0106] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, any equivalent modifications or alterations accomplished by a person of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A software development and testing method, characterized in that: The method comprises: Establishing a connection with a test environment in response to a received performance test instruction, wherein the performance test instruction is used to trigger a performance test of a program code of the software to be tested and a compilation success indicator is generated based on the program code; Deploying the installation package generated by compiling the program code into the test environment, performing performance testing, and obtaining target test values of performance indicators of the program code; If the target test value meets the threshold conditions of the performance indicators, the performance test result is determined to be a test pass, thereby completing the software development test. The threshold conditions include that the increment of the target test value is less than or equal to the preset increment threshold and the target test value is less than or equal to the preset upper limit threshold.

2. The software development and testing method according to claim 1, wherein: The method for obtaining the target test value includes: Periodically collecting multiple test values of performance indicators at each test point, the test environment including multiple test points, the performance indicators including central processing unit occupancy and memory occupancy, and the test values including central processing unit occupancy test values and memory occupancy test values; For each test point, determining a single-point test value of the CPU occupancy rate according to a plurality of the CPU occupancy rate test values, and determining a single-point test value of the memory occupancy rate according to a plurality of the memory occupancy rate test values; The target test value of the CPU occupancy is determined according to the multiple single-point test values of the CPU occupancy, and the target test value of the memory occupancy is determined according to the multiple single-point test values of the memory occupancy.

3. The software development and testing method according to claim 1, wherein: The method for determining the performance test results includes: Obtaining historical test values of the software to be tested, where the historical test values are test values in which the performance indicators in the last performance test meet the threshold conditions; Calculating the difference between the target test value and the historical test value to obtain the increment; If the increment is less than or equal to the increment threshold, and the target test value is less than or equal to the upper threshold, determining that the performance test result is a test pass; If the increment is greater than the increment threshold, and / or the target test value is greater than the upper threshold, then the performance test result is determined to be a test failure.

4. The software development and testing method according to claim 3, wherein: The method for determining the performance test results also includes: Determining a code type of the program code according to a code type identification field in the program code, the code type including a vulnerability repair type and a function extension type; If the program code is of the vulnerability repair type, determining the performance test result according to a first incremental threshold, the upper threshold, the target test value, and the historical test value, the incremental threshold including the first incremental threshold; If the program code is of the functional extension type, the performance test result is determined according to a second incremental threshold, the upper limit threshold, the target test value and the historical test value, and the incremental threshold also includes the second incremental threshold.

5. The software development and testing method according to claim 4, wherein: After determining the performance test results, the method further includes: Feedback the performance test result to each test point in the test environment, and if the performance test result indicates that the test fails, perform error troubleshooting on each test point; An email is generated based on the performance test result and sent to the target email address. If the performance test result is that the test passes, the program code is reviewed for storage. If the performance test result is that the test fails, the storage operation of the program code is terminated and the program code is checked for errors.

6. The software development and testing method according to any one of claims 1 to 5, characterized in that: After completing the software development test, it also includes: Send a notification of test passing and merge the program code into the development branch repository; The target test value is used to update the historical test value of the software to be tested, and serves as a basis for calculating the increment of the test value during the next performance test of the software to be tested.

7. The software development and testing method according to claim 6, wherein: Before responding to the received performance test instruction, the method further includes: In response to a software development test instruction, performing a specification check on the program code; If the specification check fails, the software development test is terminated; if the specification check passes, a notification of passing the specification check is sent and a static code check of the program code is performed; If the static code check fails, the software development test is terminated; if the static code check passes, a notification of the static code check passing is sent, and the compilation verification of the program code is performed; If the compilation verification fails, the software development test is terminated; if the compilation verification passes, a notification of the compilation verification passing is sent and the performance test of the program code is performed.

8. A software development and testing system, characterized in that: The system comprises: a connection management module, configured to establish a connection with a test environment in response to a received performance test instruction, wherein the performance test instruction is configured to trigger a performance test of a program code of the software to be tested and is generated based on a compilation success flag of the program code; A performance testing module, configured to deploy the installation package generated by compiling the program code into the test environment, perform performance testing, and obtain target test values of performance indicators of the program code; A performance analysis module is used to determine that the performance test result is a test pass if the target test value meets the threshold conditions of the performance indicator, thereby completing the software development test. The threshold conditions include that the increment of the target test value is less than or equal to the preset increment threshold and the target test value is less than or equal to the preset upper limit threshold.

9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the software development and testing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the software development and testing method according to any one of claims 1 to 7.

Images