Multi-dimensional analysis model construction and automatic verification method oriented to DevOps self-evaluation
By constructing a full-dimensional DevOps database and using the entropy weight method to calculate scores, the problems of intelligence and accuracy in DevOps self-assessment analysis were solved, enabling accurate assessment and improvement verification of the R&D and operations teams.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-13
- Publication Date
- 2026-03-17
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing DevOps self-assessment and analysis technologies rely on manual inspection, which cannot achieve intelligent assessment throughout the entire process, cannot dynamically adapt to industry standards, and cannot provide accurate improvement references for R&D and operations teams, resulting in low applicability of self-assessment.
We build a comprehensive DevOps database that includes R&D and operations process data from all R&D and operations teams in the industry. We eliminate invalid data through range convergence analysis and use entropy weighting to calculate stage and overall scores, and automatically verify the improvement effects.
It improves the accuracy and rationality of DevOps self-assessment analysis, can dynamically adapt to industry standards, provide accurate improvement references, ensure that assessment results are in line with industry averages, and improve the effectiveness of improvements.
Smart Images
Figure CN121682131A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of DevOps self-evaluation analysis technology, specifically to a method for constructing and automatically validating a multi-dimensional analysis model for DevOps self-evaluation. Background Technology
[0002] DevOps self-assessment and analysis technology refers to a set of data-driven methodologies and tools that allow development and operations teams to automatically, objectively, and continuously measure the capabilities and maturity of their software delivery processes, identify areas for improvement through multi-dimensional analysis, and ultimately form a verifiable improvement loop.
[0003] Existing DevOps self-assessment and analysis technologies rely too heavily on manual verification, failing to achieve truly intelligent end-to-end assessment. Furthermore, they typically use preset thresholds for judgment, making it impossible to dynamically adapt to the overall industry level. Moreover, existing DevOps self-assessment and analysis technologies cannot accurately pinpoint shortcomings within the R&D and operations teams, preventing them from making precise and targeted improvements to their R&D and operations processes. After improvements are implemented, their effectiveness cannot be verified. These existing technologies also suffer from insufficient intelligence during the assessment process and a lack of accurate reference data for R&D and operations teams' improvements, resulting in low applicability of DevOps self-assessment. Summary of the Invention
[0004] This invention aims to at least partially address one of the technical problems in existing technologies. It constructs a comprehensive DevOps database, incorporating stage data from all R&D and operations teams across the industry in each R&D and operations process. Then, it divides the stage data into auxiliary ranges and constructs a range convergence analysis diagram. Based on this diagram, it performs range convergence processing on the stage data in the comprehensive DevOps database to obtain effective data. Next, it calculates normalized data for the stage data based on this effective data, and uses the entropy weight method to calculate and further analyze the stage and comprehensive scores of the R&D and operations teams in each R&D and operations process. Finally, it evaluates the R&D and operations teams' DevOps processes based on these scores. Finally, when an R&D and operations team improves any R&D and operations process, it automatically verifies the effectiveness of the improvement. This addresses the problem that existing DevOps self-assessment analysis technologies lack intelligence during the assessment process and cannot provide accurate reference data for R&D and operations team improvements, resulting in low applicability of DevOps self-assessment.
[0005] To achieve the above objectives, this application provides a method for constructing and automatically validating a multi-dimensional analysis model for DevOps self-assessment, including the following steps: Build a DevOps full-dimensional database to collect stage data from all R&D and operations teams in the industry in every R&D and operations process; Perform range convergence processing on the stage data in the DevOps full-dimensional database, remove invalid stage data from the DevOps full-dimensional database, and obtain valid data; Normalized data based on effective data calculation stage data from the DevOps full-dimensional database; By combining normalized data with the entropy weight method, the stage score and comprehensive score of the R&D and operations team in each R&D and operations process are calculated. At the same time, the DevOps process of the R&D and operations team is evaluated based on the stage score and comprehensive score. When the R&D operations team makes improvements to any R&D operations process, the system automatically verifies whether the improvements are effective.
[0006] Furthermore, a comprehensive DevOps database is constructed, encompassing data from all R&D and operations teams across the industry, covering each stage of the R&D and operations process. This includes the following sub-steps: Build a DevOps full-dimensional database to collect stage data from all R&D and operations teams in the industry in every R&D and operations process; The R&D and operation process includes the planning phase, code phase, build phase, testing phase, deployment phase, operation phase, and monitoring phase. The phase data includes planning data, code data, build data, test data, deployment data, operation data, and monitoring data; Each of the aforementioned stage data also includes a different number of stage subdivision data.
[0007] Furthermore, the process of narrowing down the stage data in the DevOps full-dimensional database and removing invalid stage data includes the following sub-steps: Divide the stage sub-data into auxiliary ranges and construct a range convergence analysis diagram; Based on the range convergence analysis diagram, the stage data in the DevOps full-dimensional database is processed to obtain effective data.
[0008] Furthermore, dividing the stage-specific sub-data within the stage data into auxiliary ranges and constructing a range convergence analysis plot includes the following sub-steps: When analyzing data at any stage, name it the data to be analyzed; Establish a one-dimensional coordinate system with the data to be analyzed as the X-axis, and name it the Initial Analysis Chart. Enter the data to be analyzed from the DevOps full-dimensional database into the Initial Analysis Chart. Perform cluster analysis on the initial analysis graph to obtain different initial analysis clusters. Count the number of data to be analyzed in the initial analysis clusters and name them as cluster data volume. Obtain the initial analysis cluster corresponding to the largest cluster data volume and name it as cluster cluster. Calculate the average value of the data to be analyzed in the cluster cluster and name it as cluster data value. Data to be analyzed that is less than the set data value is named the first data, and data to be analyzed that is greater than or equal to the set data value is named the second data. Obtain the range of the first data in the DevOps full-dimensional database, name it the first range, and divide the first range evenly into a first number of ranges, name them the first sub-ranges. Number the first sub-ranges in ascending order, using the symbol DAR. n This indicates that n is a non-zero natural number and n is the index of DAR; Obtain the range of the second data in the DevOps full-dimensional database, name it the second range, and divide the second range evenly into a first number of ranges, name them the second sub-ranges. Number the second sub-ranges in ascending order, using the symbol DBR. n express; Statistics on DevOps across all dimensions are in DAR n The number of data to be analyzed, denoted as DAQ n The DevOps full-dimensional database is in the DBR (DevOps Blog) category. n The amount of data to be analyzed is denoted as DBQ. n ; With n as the X-axis, and DAQ respectively n and DBQ n Establish a two-dimensional coordinate system for the Y-axis, named the first range convergence analysis plot and the second range convergence analysis plot, and then define the DAQ... n Enter the first range convergence analysis diagram according to n, and then use DBQ. n Enter the second range convergence analysis diagram according to n.
[0009] Furthermore, performing range convergence processing on the stage data in the DevOps full-dimensional database based on the range convergence analysis diagram includes the following sub-steps: When analyzing the first or second range convergence analysis diagram, name it the convergence diagram to be analyzed, and name the coordinate points in the convergence diagram to be analyzed the convergence points to be analyzed. Connect adjacent convergence points to be analyzed with line segments to obtain convergence lines to be analyzed. Obtain the slope of the convergence lines to be analyzed and name it the convergence slope to be analyzed. Name the convergence line to be analyzed corresponding to the maximum value of the convergence slope to be analyzed as the effective separation line. Mark the two convergence points to be analyzed connected by the effective separation line as PH1 and PH2 respectively. DAQ of PH1 and PH2 n or DBQ n Merge the data to obtain a new range, named the separation range. Construct a two-dimensional coordinate system with the data to be analyzed as the horizontal axis and the vertical axis as meaningless, named the separation analysis chart. Enter the data to be analyzed within the separation range into the separation analysis chart. When entering the data, enter the data to be analyzed according to the horizontal axis and randomly distribute it on the vertical axis. Name the coordinate points in the separation analysis chart as separation analysis points. Construct an infinitely long straight line perpendicular to the horizontal axis and name it the separation auxiliary line. Starting from the horizontal axis as 0, move the separation auxiliary line horizontally along the positive direction of the horizontal axis and count the number of separation analysis points on the separation auxiliary line in real time, which is named the upper line point count. The number of non-zero online points is counted and named as the number of valid online points. Cluster analysis is performed on the number of valid online points, and the clustering coefficient is adjusted so that two clusters are generated, named the first separating cluster and the second separating cluster respectively. The average number of online points in the first separating cluster is less than the average number of online points in the second separating cluster. Mark the value of the data to be analyzed corresponding to the number of points on the upper line as FV. Find the two FVs that are closest to each other in the first separation cluster and the second separation cluster, and mark them as FHV1 and FHV2 respectively. Calculate (FHV1+FHV2) / 2 to get the convergence value. If the convergence plot to be analyzed is a first-range convergence plot, then the data to be analyzed that is smaller than the convergence value is removed. If the convergence plot to be analyzed is a second-range convergence plot, then the data to be analyzed that is larger than the convergence value is removed. The data to be analyzed that are finally retained are named as valid data. The data of each stage are analyzed to obtain all the valid data.
[0010] Furthermore, the normalized data for the effective data calculation stage based on the DevOps full-dimensional database is specifically obtained by calculating the effective data using a normalization algorithm.
[0011] Furthermore, by combining normalized data with the entropy weight method, the stage score and comprehensive score of the R&D operations team in each R&D operations process are calculated. Simultaneously, the DevOps process of the R&D operations team is evaluated based on the stage score and comprehensive score, including the following sub-steps: The normalized data was calculated using the entropy weight method, and the stage scores and comprehensive scores of the R&D and operations team in each R&D and operations process were further analyzed. The DevOps processes of the R&D and operations team are evaluated based on phase scores and overall scores.
[0012] Furthermore, the normalized data is calculated and analyzed using the entropy weight method, and the stage scores and overall scores of the R&D and operations team in each R&D and operations process are further analyzed, including the following sub-steps: The entropy weight method is used to calculate the normalized data of all stage sub-data in the same stage data to obtain the sub-weight of each stage sub-data in the stage data. The stage data is numbered using the symbol PD. i This indicates that, where i is a non-zero natural number and i is the PD index, the PD index is used to number the subdivided data in the PD data. i The j-th stage subdivision data in the model is labeled as PSD(i,j), where j is a non-zero natural number and (i,j) is the index of PSD. The subdivision weight of PSD(i,j) is labeled as SK(i,j), and the normalized data of PSD(i,j) is labeled as YSD(i,j). Through formula Calculate the stage score for the stage data, where F is the stage score and max() is the maximum value operator, and PD is the stage score. i The stage score is marked as PF i ; The stage scores of all stage data are calculated using the entropy weight method to obtain stage weights, and the PD is then assigned to each stage. i The stage weight is labeled K. i ; Through formula The overall score of the research and development team is calculated.
[0013] Furthermore, the evaluation of the R&D team's DevOps process based on phase scores and overall scores includes the following sub-steps: The manual-verified stage data is obtained from big data and stage scores and comprehensive scores are calculated and named stage historical scores and comprehensive historical scores, respectively. The range of unqualified stage historical scores is calculated and named stage unqualified range. The range of unqualified comprehensive historical scores is calculated and named comprehensive unqualified range. The maximum value of stage unqualified range is named stage unqualified threshold. The maximum value of comprehensive unqualified range is named comprehensive unqualified threshold. If the R&D team's stage score is less than or equal to the stage non-compliance threshold, then the stage data corresponding to the R&D team's stage score is marked as non-compliance; otherwise, the stage data corresponding to the R&D team's stage score is marked as compliance. If the overall score of the R&D and operations team is less than or equal to the overall failure threshold, the team's overall R&D and operations capabilities are marked as unqualified; otherwise, the team's overall R&D and operations capabilities are marked as qualified.
[0014] Furthermore, when the R&D operations team improves any R&D operations process, the automated verification of whether the improvement is effective is as follows: when the R&D operations team improves any R&D operations process, it determines whether the corresponding stage score of the R&D operations process increases. If so, it outputs an effective improvement signal; otherwise, it outputs an invalid improvement signal.
[0015] The beneficial effects of this invention are as follows: This invention constructs a DevOps full-dimensional database, which includes stage data from all R&D and operations teams in the industry for each R&D and operations process in big data. Then, it divides the stage data into auxiliary ranges and constructs a range convergence analysis diagram. Based on the range convergence analysis diagram, it performs range convergence processing on the stage data in the DevOps full-dimensional database to obtain effective data. The advantage is that, in the stage data of all R&D and operations teams in the industry for each R&D and operations process in big data, due to the uneven level of different R&D and operations teams, there are some data that are far below or far above the average level. Such data usually has no reference value. If they are included in the reference range, the final evaluation result will deviate from the normal judgment standard of the industry. However, this invention performs range convergence processing on all stage data, which can remove data that is far below or far above the industry average level, so that the threshold for the final evaluation can be more in line with the industry average level, thereby improving the accuracy and rationality of DevOps self-assessment analysis. This invention calculates normalized data for each stage of DevOps based on valid data from a full-dimensional DevOps database. It then uses the entropy weight method to calculate and further analyze the stage and comprehensive scores of the R&D and operations teams in each R&D and operations process. Based on these stage and comprehensive scores, the invention evaluates the DevOps processes of the R&D and operations teams. Finally, when the R&D and operations teams improve any R&D and operations process, the invention automatically verifies the effectiveness of these improvements. The advantage lies in the fact that stage and comprehensive scores reflect the capabilities of the R&D and operations teams at each R&D and operations stage or the entire team during R&D and operations. Higher stage and comprehensive scores indicate stronger capabilities. Furthermore, the R&D and operations teams can understand whether they have weaknesses in a particular R&D and operations stage based on the stage scores, allowing them to improve that stage. Simultaneously, the invention assesses the effectiveness of the R&D and operations teams' improvements based on the stage scores, thus improving the accuracy and effectiveness of DevOps self-assessment analysis. Attached Figure Description
[0016] Figure 1 This is a flowchart illustrating the steps of the method of the present invention; Figure 2 This is a schematic diagram of the first range convergence analysis diagram of the present invention; Figure 3 This is a schematic diagram of the second range convergence analysis diagram of the present invention; Figure 4 This is a schematic diagram of the convergence line to be analyzed in the second range convergence analysis diagram of the present invention; Figure 5 This is a schematic diagram of the separation analysis diagram of the present invention; Figure 6 This is a schematic diagram of the separation auxiliary line of the present invention. Detailed Implementation
[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0018] Example 1, please refer to Figure 1 As shown, this application provides a multi-dimensional analysis model construction and automated verification method for DevOps self-assessment, including the following steps: Step S1 involves building a comprehensive DevOps database, which includes stage data from all R&D and operations teams across the industry in each R&D and operations process. Step S1 includes the following sub-steps: Step S101: Build a DevOps full-dimensional database to collect stage data from all R&D and operations teams in the industry in each R&D and operations process from big data. Step S102, the R&D and operation process includes the planning stage, code stage, build stage, testing stage, deployment stage, operation stage and monitoring stage; Step S103, the stage data includes planning data, code data, build data, test data, deployment data, operation data, and monitoring data; Step S104, each stage data also includes a different number of stage subdivision data; In practice, stage data refers to the specific data generated during the R&D and operation process. For example, planning data is the data generated during the planning stage. Each different stage data also includes different stage sub-data. For example, in the construction data, there are various stage sub-data such as the number of constructions, the construction success rate, and the average construction time. This embodiment uses the number of constructions, the number of successful constructions, and the average construction time as examples to illustrate the subsequent analysis process. All stage data can be directly obtained through the various control systems used by the R&D and operation team, and this embodiment will not provide further details.
[0019] Step S2 involves performing range convergence processing on the stage data in the DevOps full-dimensional database, removing invalid stage data to obtain valid data; Step S2 includes the following sub-steps: Step S201: Divide the stage subdivision data in the stage data into auxiliary ranges and construct a range convergence analysis diagram; Step S201 includes the following sub-steps: Step S201.1: When analyzing data from any stage subdivision, name it as the data to be analyzed; Step S201.2: Establish a one-dimensional coordinate system with the data to be analyzed as the X-axis, name it the initial analysis chart, and enter the data to be analyzed from the DevOps full-dimensional database into the initial analysis chart; Step S201.3: Perform cluster analysis on the initial analysis graph to obtain different initial analysis clusters. Count the number of data to be analyzed in the initial analysis clusters and name them as cluster data volume. Obtain the initial analysis cluster corresponding to the largest cluster data volume and name it as cluster cluster. Calculate the average value of the data to be analyzed in the cluster cluster and name it as cluster data value. Step S201.4: Name the data to be analyzed that is less than the set data value as the first data, and name the data to be analyzed that is greater than or equal to the set data value as the second data; In practice, for example, when the data to be analyzed is the build success rate, a one-dimensional coordinate system is constructed and the build success rate from the DevOps full-dimensional database is entered into the one-dimensional coordinate system to obtain the initial analysis graph. The initial analysis clusters are obtained through existing cluster analysis, and then the clusters are extracted. The clusters contain the most build success rates, which means that most R&D teams in the industry are at the cluster level in terms of the build success rate sub-data, close to the industry average. Based on this, the first data and the second data are divided, which represent build success rates that are below and above the industry average, respectively.
[0020] Step S201.5: Obtain the range of the first data in the DevOps full-dimensional database, name it the first range, divide the first range evenly into a first number of ranges, name them the first sub-ranges, and number the first sub-ranges in ascending order, using the symbol DAR. n This indicates that n is a non-zero natural number and n is the index of DAR; Step S201.6: Obtain the range of the second data in the DevOps full-dimensional database, name it the second range, divide the second range evenly into a first number of ranges, name them the second sub-ranges, and number the second sub-ranges in ascending order, using the symbol DBR. n express; Step S201.7: Statistically analyze the DevOps full-dimensional database to identify those in DAR (Digital Availability). n The number of data to be analyzed, denoted as DAQ n The DevOps full-dimensional database is in the DBR (DevOps Blog) category. n The amount of data to be analyzed is denoted as DBQ. n ; Please see Figures 2 to 3 As shown, in step S201.8, with n as the X-axis, and using DAQ... n and DBQ n Establish a two-dimensional coordinate system for the Y-axis, named the first range convergence analysis plot and the second range convergence analysis plot, and then define the DAQ... n Enter the first range convergence analysis diagram according to n, and then use DBQ. n Enter the second range convergence analysis diagram according to n; In practice, the first range is statistically determined to be [30%, 65%], and the second range is [60%, 95%]. The first number is set by the evaluator. The larger the first number, the more accurate the effective data for analysis. In this embodiment, the first number is set to 10, that is, the first range and the second range are respectively divided into 10 first sub-ranges and 10 second sub-ranges. Taking the second range as an example, it is divided into DBR1 to DBR 10 The percentages are [60%, 63.5%], [63.5%, 67%], [67%, 70.5%], [70.5%, 74%], [74%, 77.5%], [77.5%, 81%], [81%, 84.5%], [84.5%, 88%], [88%, 91.5%], and [91.5%, 95%], respectively. Then, the DAQ... n and DBQ n Statistical analysis was performed to construct the first range convergence analysis diagram, as shown below. Figure 2 As shown, the second range convergence analysis diagram is as follows: Figure 3 As shown.
[0021] Step S202: Based on the range convergence analysis diagram, perform range convergence processing on the stage data in the DevOps full-dimensional database to obtain effective data; Step S202 includes the following sub-steps: Step S202.1: When analyzing the first range convergence analysis diagram or the second range convergence analysis diagram, name it as the convergence diagram to be analyzed, and name the coordinate points in the convergence diagram to be analyzed as convergence points to be analyzed. Please see Figure 4As shown, in step S202.2, adjacent convergence points to be analyzed are connected by line segments to obtain convergence lines to be analyzed. The slope of the convergence lines to be analyzed is obtained and named the convergence slope to be analyzed. The convergence line to be analyzed corresponding to the maximum value of the convergence slope to be analyzed is named the effective separation line. The two convergence points to be analyzed connected by the effective separation line are marked as PH1 and PH2 respectively. In specific implementation, taking the second-range convergence analysis diagram as the convergence diagram to be analyzed as an example, the second-range convergence analysis diagram is obtained by connecting the diagrams as follows: Figure 4 As shown, Figure 4 The line segment between any two convergence points to be analyzed is the convergence line to be analyzed. Figure 4 It is easy to see that the slope of the convergence line to be analyzed between 8 and 9 on the X-axis is the largest, which means that the difference in the number of data to be analyzed between these two ranges is the largest. That is, PH1 is the convergence point to be analyzed corresponding to DBR8, and PH2 is the convergence point to be analyzed corresponding to DBR9. Since some data that do not have reference value are usually outside the majority of the data to be analyzed, and such data usually only accounts for a small part, their density in distribution is quite different from that of the majority of the data to be analyzed. That is, the difference in the number of data to be analyzed between the two ranges is the largest, but further detailed analysis of the convergence values is needed to complete the optimal range convergence processing.
[0022] Please see Figure 5 As shown, in step S202.3, the DAQ of PH1 and PH2 is... n or DBQ n Merge the data to obtain a new range, named the separation range. Construct a two-dimensional coordinate system with the data to be analyzed as the horizontal axis and the vertical axis as meaningless, named the separation analysis chart. Enter the data to be analyzed within the separation range into the separation analysis chart. When entering the data, enter the data to be analyzed according to the horizontal axis and randomly distribute it on the vertical axis. Name the coordinate points in the separation analysis chart as separation analysis points. Please see Figure 6 As shown, in step S202.4, construct a straight line that is perpendicular to the horizontal axis and infinitely long, named the separation auxiliary line. Starting from the horizontal axis as 0, move the separation auxiliary line horizontally along the positive direction of the horizontal axis, and count the number of separation analysis points on the separation auxiliary line in real time, named the upper line point count. In practice, PH1 and PH2 may be in either the first or second range convergence analysis diagram. If the convergence points to be analyzed are PH1 and PH2, and PH1 and PH2 belong to the first range convergence analysis diagram, then the DAQ of PH1 and PH2 will be... n If PH1 and PH2 belong to the second range convergence analysis plot, then merge the DBQ of PH1 and PH2. n Combined, since PH1 is [84.5%, 88%] and PH2 is [88%, 91.5%], all values are DAQ.n Therefore, the combined separation range is [84.5%, 91.5%], and the resulting separation analysis graph is shown below. Figure 5 As shown, the horizontal axis in the separation analysis chart represents the separation range, while the vertical axis has no meaning. For example, if a certain data point to be analyzed is 86%, a separation analysis point is randomly generated at the horizontal axis position equal to 86 during data entry. The vertical axis can be arbitrarily generated; this is to facilitate the observation of the upper limit point count in this embodiment, and to construct the separation auxiliary line as shown. Figure 6 As shown, at this time there is only one separation analysis point on the separation auxiliary line, that is, the number of upper line points is 1.
[0023] Step S202.5: Count the number of non-zero online points and name them as the number of valid online points. Perform cluster analysis on the number of valid online points and adjust the clustering coefficients so that two clusters are generated in the end, named the first separation cluster and the second separation cluster respectively. The average number of online points in the first separation cluster is less than the average number of online points in the second separation cluster. Step S202.6: Mark the value of the data to be analyzed corresponding to the number of points on the upper line as FV, find the two FVs that are closest to each other in the first separation cluster and the second separation cluster, and mark them as FHV1 and FHV2 respectively. Calculate (FHV1+FHV2) / 2 to obtain the convergence value. Step S202.7: If the convergence plot to be analyzed is a first-range convergence plot, then the data to be analyzed that is smaller than the convergence value is removed. If the convergence plot to be analyzed is a second-range convergence plot, then the data to be analyzed that is larger than the convergence value is removed. The data to be analyzed that are finally retained are named as valid data. The data of each stage are analyzed to obtain all the valid data. In practice, the proportion of data without reference value is very small, resulting in a large gap between the number of online data points and the effective data. In this case, cluster analysis can be used to filter out the data without reference value. By adjusting the clustering coefficients in the cluster analysis, the results are divided into two clusters: one representing effective data and the other representing data without reference value. For example... Figure 6 The upper line shown is 1, and the separation auxiliary line is at 92.6% of the horizontal axis, that is, the FV is 92.6%. Find the two FVs that are closest to each other in the first separation cluster and the second separation cluster, that is, the two data to be analyzed that are closest between the effective data and the data that have no reference value. Finally, calculate the convergence value, and then distinguish them according to the convergence value to obtain the effective data.
[0024] Step S3: Calculate normalized data for the stage data based on the effective data in the DevOps full-dimensional database; calculate the normalized data by using a normalization algorithm on the effective data. In practice, since the normalization algorithm is an existing method for quantifying and unifying data, it will not be described in detail in this embodiment.
[0025] Step S4 involves calculating the stage score and overall score of the R&D operations team in each R&D operations process using normalized data and the entropy weight method. Simultaneously, the DevOps process of the R&D operations team is evaluated based on the stage score and overall score. Step S4 includes the following sub-steps: Step S401: Calculate the normalized data using the entropy weight method and further analyze the stage scores and comprehensive scores of the R&D and operations team in each R&D and operations process; Step S401 includes the following sub-steps: Step S401.1: Calculate the normalized data of all stage subdivision data in the same stage data using the entropy weight method to obtain the subdivision weight of the stage subdivision data in the stage data. Step S401.2: Number the stage data using the symbol PD. i This indicates that, where i is a non-zero natural number and i is the PD index, the PD index is used to number the subdivided data in the PD data. i The j-th stage subdivision data in the model is labeled as PSD(i,j), where j is a non-zero natural number and (i,j) is the index of PSD. The subdivision weight of PSD(i,j) is labeled as SK(i,j), and the normalized data of PSD(i,j) is labeled as YSD(i,j). Step S401.3, using the formula Calculate the stage score for the stage data, where F is the stage score and max() is the maximum value operator, and PD is the stage score. i The stage score is marked as PF i ; In specific implementation, the subdivision weights are calculated using the existing entropy weight method. For example, in the construction data listed in this embodiment, there are construction counts, construction success rates, and average construction times. The entropy weight method calculates the subdivision weights corresponding to the construction counts, construction success rates, and average construction times to be 0.2274, 0.4838, and 0.2888, respectively. These are numbered PD1 to PD7, representing planning data, code data, construction data, test data, deployment data, operation data, and monitoring data, respectively. Construction data is PD3, and the construction counts, construction success rates, and average construction times are PSD(3,1), PSD(3,2), and PSD(3,3), respectively, with weights of 0.2274, 0.4838, and 0.2888. 838 and 0.2888 are SK(3,1), SK(3,2), and SK(3,3). For example, in a certain build data, the normalized data of the number of builds, the success rate of builds, and the average build time are 0.56, 0.83, and 0.42, respectively, which correspond to YSD(3,1), YSD(3,2), and YSD(3,3). However, some stage sub-data are not necessarily better the larger they are, such as the number of builds and the average build time. Therefore, they can be set to negative numbers in the calculation. That is, YSD(3,1) is actually -0.56, and YSD(3,3) is actually -0.42. The calculated PF3 is 0.152914, which is the stage score of the build stage.
[0026] Step S401.4: Calculate the stage scores for all stage data using the entropy weight method to obtain the stage weights, and then assign the PD... i The stage weight is labeled K. i ; Step S401.5, using the formula Calculate the overall score of the research and development team; In practice, the comprehensive scores of the R&D and operations team's planning, coding, building, testing, deployment, operation, and monitoring phases are calculated in the same way. Since the analysis process is similar, this embodiment will not be described in detail.
[0027] Step S402: Evaluate the DevOps process of the R&D team based on the stage score and the overall score; Step S402 includes the following sub-steps: Step S402.1: Obtain manually inspected stage data from big data and calculate stage score and comprehensive score, which are named stage historical score and comprehensive historical score, respectively. Calculate the range of unqualified stage historical scores and name it stage unqualified range. Calculate the range of unqualified comprehensive historical scores and name it comprehensive unqualified range. Name the maximum value of the stage unqualified range as stage unqualified threshold and name the maximum value of the comprehensive unqualified range as comprehensive unqualified threshold. Step S402.2: If the stage score of the R&D team is less than or equal to the stage non-compliance threshold, then mark the stage data corresponding to the stage score of the R&D team as non-compliance; otherwise, mark the stage data corresponding to the stage score of the R&D team as qualified. Step S402.3: If the overall score of the R&D and operations team is less than or equal to the overall unqualified threshold, then the overall R&D and operations team's R&D and operations capabilities are marked as unqualified; otherwise, the overall R&D and operations team's R&D and operations capabilities are marked as qualified. In practice, manually inspected stage data is obtained from big data and stage scores and comprehensive scores are calculated. Stage non-compliance thresholds and comprehensive non-compliance thresholds are then calculated. Although manual inspection is also used, this manual inspection does not need to be performed in this embodiment. Instead, it is obtained from existing data in big data to analyze stage non-compliance thresholds and comprehensive non-compliance thresholds. Furthermore, no further manual inspection is required. The entire evaluation process is completed intelligently.
[0028] Step S5: When the R&D operations team improves any R&D operations process, automatically verify whether the improvement is effective; when the R&D operations team improves any R&D operations process, determine whether the stage score corresponding to the R&D operations process increases. If so, output an effective improvement signal; otherwise, output an invalid improvement signal. In practice, the R&D team can identify its shortcomings based on the stage score and make improvements accordingly. The effectiveness of these improvements can be gauged by whether the stage score increases.
[0029] Example 2: This application provides an electronic device, which may include a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The memory stores computer-readable instructions, and the processor can call these instructions. When the processor executes a computer-readable instruction, it performs steps such as those in the DevOps self-assessment multi-dimensional analysis model construction and automated verification method to achieve the following functions: constructing a DevOps full-dimensional database and including stage data; performing range convergence processing on the stage data in the DevOps full-dimensional database to obtain valid data; calculating normalized data for stage data based on the valid data in the DevOps full-dimensional database; calculating the stage score and comprehensive score of the R&D and operations team in each R&D and operations process, and simultaneously evaluating the R&D and operations team's DevOps processes based on the stage score and comprehensive score; and automatically verifying the effectiveness of improvements to the R&D and operations processes.
[0030] Furthermore, when the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0031] Example 3: This application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the multi-dimensional analysis model construction and automated verification method for DevOps self-assessment provided by the above methods. This method includes: constructing a DevOps full-dimensional database and collecting stage data; performing range convergence processing on the stage data in the DevOps full-dimensional database to obtain effective data; calculating normalized data of stage data based on the effective data in the DevOps full-dimensional database; calculating the stage score and comprehensive score of the R&D and operations team in each R&D and operations process, and evaluating the DevOps process of the R&D and operations team based on the stage score and comprehensive score; and automatically verifying whether the improvement of the R&D and operations process is effective.
[0032] Example 4: This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program performs the steps described above in the method for constructing and automatically verifying a multi-dimensional analysis model for DevOps self-assessment, to achieve the following functions: constructing a DevOps full-dimensional database and including stage data; performing range convergence processing on the stage data in the DevOps full-dimensional database to obtain effective data; calculating normalized data of stage data based on the effective data in the DevOps full-dimensional database; calculating the stage score and comprehensive score of the R&D and operations team in each R&D and operations process, and simultaneously evaluating the DevOps process of the R&D and operations team based on the stage score and comprehensive score; and automatically verifying whether the improvement of the R&D and operations process is effective.
[0033] Based on the above description of the embodiments, the embodiments of the present invention can be provided as methods, systems, or computer program products. Based on this understanding, the above technical solutions, in essence or in terms of their contribution to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or certain parts of the embodiments.
[0034] In the embodiments provided in this application, it should be understood that the disclosed system or method can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules or units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces. The indirect coupling or communication connection between systems, modules, and units may be electrical, mechanical, or other forms.
[0035] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A method for constructing and automatically verifying a multi-dimensional analysis model for DevOps self-assessment, characterized in that, The method comprises the following steps: A DevOps full-dimension database is constructed to record stage data of all research and operation teams in the industry in each research and operation process in big data. The stage data in the DevOps full-dimension database is subjected to range convergence processing to eliminate invalid stage data in the DevOps full-dimension database to obtain effective data. The normalized data of the stage data is calculated based on the effective data in the DevOps full-dimension database. The stage score and the comprehensive score of the research and operation team in each research and operation process are calculated by the normalized data combined with the entropy weight method, and the DevOps process of the research and operation team is evaluated based on the stage score and the comprehensive score. When the research and operation team improves any research and operation process, it is automatically verified whether the improvement of the research and operation process is effective.
2. The method as claimed in claim 1, wherein the method is characterized by: The method of constructing a DevOps full-dimension database to record stage data of all research and operation teams in the industry in each research and operation process in big data comprises the following sub-steps: A DevOps full-dimension database is constructed to record stage data of all research and operation teams in the industry in each research and operation process in big data. The research and operation process comprises a planning stage, a code stage, a construction stage, a test stage, a deployment stage, an operation stage and a monitoring stage. The stage data comprises planning data, code data, construction data, test data, deployment data, operation data and monitoring data. Each kind of stage data further comprises different amounts of stage subdivision data.
3. The method of claim 2, wherein the method further comprises: The method of eliminating invalid stage data in the DevOps full-dimension database by range convergence processing of the stage data in the DevOps full-dimension database comprises the following sub-steps: An auxiliary range is divided for the stage subdivision data in the stage data, and a range convergence analysis graph is constructed. The stage data in the DevOps full-dimension database is subjected to range convergence processing based on the range convergence analysis graph to obtain effective data.
4. The method of claim 3, wherein the method further comprises: The method of dividing an auxiliary range for the stage subdivision data in the stage data and constructing a range convergence analysis graph comprises the following sub-steps: When any stage subdivision data is analyzed, it is named as to-be-analyzed data. A one-dimensional coordinate system is established with the to-be-analyzed data as the X axis, and is named as an initial analysis graph. The to-be-analyzed data in the DevOps full-dimension database is recorded in the initial analysis graph. The initial analysis graph is subjected to cluster analysis to obtain different initial analysis clusters, the number of to-be-analyzed data in the initial analysis clusters is counted and named as cluster data quantity, the initial analysis cluster corresponding to the maximum cluster data quantity is obtained and named as a concentrated cluster, the average value of the to-be-analyzed data in the concentrated cluster is calculated and named as a concentrated data value. Obtaining a range of the first data in the DevOps full-dimension database, named as a first range, dividing the first range into a first number of ranges, named as first sub-ranges, numbering the first sub-ranges in ascending order, and representing the first sub-ranges by a symbol DAR n , wherein n is a non-zero natural number and n is the serial number of DAR. Obtain the range of the second data in the DevOps full-dimension database, named as a second range, uniformly divide the second range into a first number of ranges, named as second sub-ranges, number the second sub-ranges in ascending order, and obtain the second sub-range corresponding to the second data through a symbol DBR n representing; counting the number of data to be analyzed in the DAR n n counting the number of data to be analyzed in the DBR n n counting the number of data to be analyzed in the DBR A two-dimensional coordinate system is established with n as the X axis and DAQ n and DBQ n as the Y axis, and is named the first range convergence analysis graph and the second range convergence analysis graph. DAQ n is entered into the first range convergence analysis graph according to n, and DBQ n is entered into the second range convergence analysis graph according to n.
5. The DevOps-oriented self-evaluation multi-dimensional analysis model construction and automated verification method according to claim 4, characterized in that, The to-be-analyzed data less than the concentrated data value is named as first data, and the to-be-analyzed data greater than or equal to the concentrated data value is named as second data. The method of subjecting the stage data in the DevOps full-dimension database to range convergence processing based on the range convergence analysis graph comprises the following sub-steps: When the first range convergence analysis graph or the second range convergence analysis graph is analyzed, it is named as a to-be-analyzed convergence graph, and the coordinate points in the to-be-analyzed convergence graph are named as to-be-analyzed convergence points. The adjacent to-be-analyzed convergent points are connected by line segments to obtain a to-be-analyzed convergent line, the slope of the to-be-analyzed convergent line is obtained and named as a to-be-analyzed convergent slope, a to-be-analyzed convergent line corresponding to the maximum value of the to-be-analyzed convergent slope is named as a separation effective line, and two to-be-analyzed convergent points connected by the separation effective line are marked as PH1 and PH2, respectively; DAQ of PH1 and PH2 n or DBQ n Merge the data to obtain a new range, named the separation range. Construct a two-dimensional coordinate system with the data to be analyzed as the horizontal axis and the vertical axis as meaningless, named the separation analysis chart. Enter the data to be analyzed within the separation range into the separation analysis chart. When entering the data, enter the data to be analyzed according to the horizontal axis and randomly distribute it on the vertical axis. Name the coordinate points in the separation analysis chart as separation analysis points. A straight line perpendicular to the horizontal axis and infinite in length is constructed and named as a separation auxiliary line, the separation auxiliary line is moved horizontally along the positive direction of the horizontal axis starting from 0, the number of separation analysis points on the separation auxiliary line is counted in real time and named as an upper line point number; The upper line point number that is not zero is counted and named as an upper line effective point number, the upper line effective point number is subjected to cluster analysis, and a cluster coefficient is adjusted so that two clusters are finally generated and named as a first separation cluster and a second separation cluster, respectively, and the average upper line point number of the first separation cluster is less than that of the second separation cluster; A value of to-be-analyzed data corresponding to the upper line point number is marked as FV, two FVs closest to each other in the first separation cluster and the second separation cluster are found and marked as FHV1 and FHV2, respectively, (FHV1+FHV2) / 2 is calculated to obtain a convergent value; If the to-be-analyzed convergent graph is a first range convergent analysis graph, to-be-analyzed data less than the convergent value is removed, if the to-be-analyzed convergent graph is a second range convergent analysis graph, to-be-analyzed data greater than the convergent value is removed, and finally retained to-be-analyzed data is named as effective data, each stage subdivision data is analyzed to obtain all effective data.
6. The DevOps-oriented self-evaluation multi-dimensional analysis model construction and automated verification method according to claim 5, characterized in that, The normalized data of the stage data calculated based on the effective data in the DevOps full-dimension database is specifically calculated by a normalization algorithm on the effective data to obtain the normalized data.
7. The DevOps-oriented self-evaluation multi-dimensional analysis model construction and automated verification method according to claim 6, characterized in that, The stage score and the comprehensive score of the research and operation team in each research and development operation process are calculated by the normalized data combined with the entropy weight method, and the DevOps process of the research and operation team is evaluated based on the stage score and the comprehensive score, including the following sub-steps: The stage score and the comprehensive score of the research and operation team in each research and development operation process are calculated by the normalized data combined with the entropy weight method, and the DevOps process of the research and operation team is evaluated based on the stage score and the comprehensive score. The stage score and the comprehensive score of the research and operation team in each research and development operation process are calculated by the normalized data combined with the entropy weight method, including the following sub-steps:
8. The DevOps-oriented self-evaluation multi-dimensional analysis model construction and automated verification method according to claim 7, characterized in that, The stage score and the comprehensive score of the research and operation team in each research and development operation process are calculated by the normalized data combined with the entropy weight method, and the DevOps process of the research and operation team is evaluated based on the stage score and the comprehensive score. The stage data subjected to artificial inspection is obtained from big data, and the stage score and the comprehensive score are calculated, which are named as a stage historical score and a comprehensive historical score, respectively, the range of unqualified stage historical scores is counted and named as a stage unqualified range, the range of unqualified comprehensive historical scores is counted and named as a comprehensive unqualified range, the maximum value of the stage unqualified range is named as a stage unqualified threshold, and the maximum value of the comprehensive unqualified range is named as a comprehensive unqualified threshold. The stage data is numbered by a symbol PD i , where i is a non-zero natural number and i is the serial number of PD. The stage subdivision data in the stage data is numbered, and the jth stage subdivision data in PD i is marked as PSD(i, j), where j is a non-zero natural number and (i, j) is the serial number of PSD. The subdivision weight of PSD(i, j) is marked as SK(i, j), and the normalized data of PSD(i, j) is marked as YSD(i, j). Through formula Calculate the stage score for the stage data, where F is the stage score and max() is the maximum value operator, and PD is the stage score. i The stage score is marked as PF i ; The stage weight is obtained by calculating the stage score of all stage data by the entropy weight method, and the stage weight of the PD i is marked as K i ; The overall score of the research team is calculated by the formula S = 0.5 * (S1 + S2 + S3 + S4 + 9. The DevOps-oriented self-evaluation multi-dimensional analysis model construction and automated verification method according to claim 8, characterized in that, If the stage score of the research and operation team is less than or equal to the stage unqualified threshold, the stage data corresponding to the stage score of the research and operation team is marked as unqualified, otherwise the stage data corresponding to the stage score of the research and operation team is marked as qualified; If the comprehensive score of the research and operation team is less than or equal to the comprehensive unqualified threshold, the research and development operation capability of the research and operation team as a whole is marked as unqualified, otherwise the research and development operation capability of the research and operation team as a whole is marked as qualified.
10. The DevOps-oriented self-evaluation multi-dimensional analysis model construction and automated verification method according to claim 9, characterized in that, When the research and operation team improves any research and development operation process, the improvement of the research and development operation process is automatically verified to be effective, specifically, when the research and operation team improves any research and development operation process, it is judged whether the stage score corresponding to the research and development operation process rises, if yes, an improvement effective signal is output, otherwise an improvement ineffective signal is output.