Change Evaluation Method and System for Application Systems

Abstract

The present invention discloses a change assessment method and system for an application system. The method includes: performing a change operation when an application system encounters a problem that cannot be fundamentally solved; introducing change assessment factors for different business types and collecting corresponding object information; determining the TF value and the occurrence probability IDF value of each change assessment factor in all change assessment factors based on the TF-IDF algorithm according to the change assessment factors; judging whether the change value of each change assessment factor before and after the change is within a preset effective range, and obtaining a TH value according to the judgment result; calculating the sum of the products of the TF value, IDF value and TH value of all change assessment factors in the application system as the robustness score of the application system, and evaluating the effectiveness of the application system after the change. Through the technical solution of the present invention, it is possible to evaluate whether the change of the application system is effective and whether the application system is running stably after the change, providing an effective reference basis for the next change.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a change assessment method for an application system and a change assessment system for an application system. Background Art

[0002] The fundamental purpose of using information technology is to support business operations, whether strategic, tactical, or operational. The external environment facing an enterprise is constantly changing, and its business operations must adapt accordingly. Consequently, the IT applications and infrastructure supporting these operations must also undergo necessary changes.

[0003] Therefore, change is essential in business operations. Implementing changes can be risky, so it's crucial to first obtain approval from business management and ensure they understand the scope and extent of the change. Preemptive notification to users and a release process are essential steps when implementing changes to avoid issues that could seriously impact user experience.

[0004] During the application system change process, there is currently no specific method to evaluate whether the application system runs stably after the change, whether it has any impact on the business, and it is also impossible to effectively evaluate the effectiveness of the change. Summary of the Invention

[0005] In response to the above problems, the present invention provides a change assessment method and system for an application system. By introducing a change assessment factor into the changed application system, object information of the change assessment factors of different business types is collected, and a robustness score is obtained by analyzing and calculating the TF value, IDF value and TH value of the change assessment factor. The robustness score is used to evaluate whether the change of the application system is effective and whether the application system runs stably after the change, thereby providing an effective reference basis for the next change.

[0006] To achieve the above object, the present invention provides a method for evaluating changes in an application system, comprising:

[0007] When the problem management module finds that an application system has a problem that cannot be fundamentally solved, it submits a change request to the change management module to perform a change operation on the application system;

[0008] According to the characteristics of the application system, corresponding change assessment factors are introduced for different business types, and the object information corresponding to each change assessment factor is collected after the change operation is completed;

[0009] According to the change assessment factors, determine the proportion TF value of each change assessment factor in all change assessment factors and the occurrence probability IDF value of each change assessment factor in all change assessment factors based on the TF-IDF algorithm;

[0010] Obtain the object information corresponding to each change assessment factor before the change operation, and determine whether the change value of the object information corresponding to each change assessment factor before and after the change is within the preset valid range, and obtain the TH value of whether the change assessment factor is valid based on the determination result;

[0011] The sum of the products of the TF value, IDF value and TH value of all change evaluation factors in the application system is calculated as the robustness score of the application system to evaluate the effectiveness of the application system after the change.

[0012] In the above technical solution, preferably, the specific process of submitting a change request to the change management module to perform a change operation on the application system includes:

[0013] The change management module completes the change operation of any aspect of the infrastructure or service within the preset interruption time according to the change request and the preset service management process.

[0014] In the above technical solution, preferably, the change evaluation factors include CPU usage, memory usage, disk IO throughput, network bandwidth utilization, number of database connections, number of database operations per unit time, page response time and interface response time.

[0015] In the above technical solution, preferably, the specific process of determining the TF value of the proportion of each change assessment factor in all change assessment factors and the IDF value of the occurrence probability of each change assessment factor in all change assessment factors based on the TF-IDF algorithm includes:

[0016] Determine the number of times w(P, T) a change assessment factor T is used to mark a service type P;

[0017] Calculate the ratio of the number of times the change assessment factor T is marked to the number of all change assessment factors of the current service type P as the TF value;

[0018] The sum of all business types and all change assessment factors is calculated, and the ratio of the sum to the sum of all types corresponding to the current change assessment factor is calculated as the IDF value.

[0019] In the above technical solution, preferably, the specific process of obtaining the object information corresponding to each change assessment factor before the change operation, determining whether the change value of the object information corresponding to each change assessment factor before and after the change is within a preset valid range, and obtaining the TH value indicating whether the change assessment factor is valid based on the determination result includes:

[0020] Collecting object information corresponding to each change assessment factor of the application system before the change operation;

[0021] For each change assessment factor, the change value of the object information before and after the change operation is compared. If the change value exceeds the preset valid range, the current change assessment factor is determined to be invalid and the TH value is set to 0. If the change value does not exceed the preset valid range, the current change assessment factor is determined to be valid and the TH value is set to 1.

[0022] In the above technical solution, preferably, the process of calculating the sum of the products of the TF value, IDF value, and TH value of all change evaluation factors in the application system as the robustness score of the application system and evaluating the effectiveness of the changed application system includes:

[0023] Calculate the product of TF value, IDF value and TH value of each change assessment factor;

[0024] Calculating the sum of the products of all change assessment factors in the application system as the robustness score of the application system;

[0025] The effectiveness of the change of the application system is evaluated according to the robustness score of the application system. The lower the robustness score, the worse the effectiveness of the change operation of the application system.

[0026] The present invention further provides an application system change assessment system, which applies the application system change assessment method disclosed in any one of the above technical solutions, including:

[0027] A change operation module is used to perform a change operation on the application system when the problem management module finds that the application system has a problem that cannot be fundamentally solved and submits a change request to the change management module;

[0028] The factor collection module is used to introduce corresponding change assessment factors for different business types based on the characteristics of the application system, and collect the object information corresponding to each change assessment factor after the change operation is completed;

[0029] A change analysis module is used to determine, based on the change assessment factors and the TF-IDF algorithm, the proportion TF value of each change assessment factor in all change assessment factors and the IDF value of the occurrence probability of each change assessment factor in all change assessment factors;

[0030] The change analysis module is further configured to obtain object information corresponding to each change assessment factor before the change operation, and determine whether the change value of the object information corresponding to each change assessment factor before and after the change is within a preset valid range, and obtain a TH value indicating whether the change assessment factor is valid based on the determination result;

[0031] The change evaluation module is used to calculate the sum of the products of the TF value, IDF value and TH value of all change evaluation factors in the application system as the robustness score of the application system, and evaluate the effectiveness of the application system after the change.

[0032] In the above technical solution, preferably, the change operation module is specifically used to:

[0033] The change management module completes the change operation of any aspect of the infrastructure or service within the preset interruption time according to the change request and the preset service management process.

[0034] In the above technical solution, preferably, the change analysis module is specifically used to:

[0035] Determine the number of times w(P, T) a change assessment factor T is used to mark a service type P;

[0036] Calculate the ratio of the number of times the change assessment factor T is marked to the number of all change assessment factors of the current service type P as the TF value;

[0037] Calculate the sum of all business types and all change assessment factors, and calculate the ratio of this sum to the sum of all types corresponding to the current change assessment factor as the IDF value;

[0038] Collecting object information corresponding to each change assessment factor of the application system before the change operation;

[0039] For each change assessment factor, the change value of the object information before and after the change operation is compared. If the change value exceeds the preset valid range, the current change assessment factor is determined to be invalid and the TH value is set to 0. If the change value does not exceed the preset valid range, the current change assessment factor is determined to be valid and the TH value is set to 1.

[0040] In the above technical solution, preferably, the change assessment module is specifically used to:

[0041] Calculate the product of TF value, IDF value and TH value of each change assessment factor;

[0042] Calculating the sum of the products of all change assessment factors in the application system as the robustness score of the application system;

[0043] The effectiveness of the change of the application system is evaluated according to the robustness score of the application system. The lower the robustness score, the worse the effectiveness of the change operation of the application system.

[0044] Compared with the existing technology, the beneficial effects of the present invention are: by introducing change assessment factors to the changed application system, collecting object information of change assessment factors of different business types, and analyzing and calculating the TF value, IDF value and TH value of the change assessment factors, a robustness score is obtained to evaluate whether the change of the application system is effective and whether the application system runs stably after the change, providing an effective reference basis for the next change. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flow chart of a method for evaluating changes in an application system disclosed in one embodiment of the present invention;

[0046] Figure 2 This is an example diagram of change assessment factors for different service types disclosed in one embodiment of the present invention;

[0047] Figure 3 A schematic diagram of modules of a change assessment system for an application system disclosed in an embodiment of the present invention.

[0048] In the figure, the corresponding relationship between each component and the reference numeral is as follows:

[0049] 11. Change operation module, 12. Factor collection module, 13. Change analysis module, 14. Change evaluation module. DETAILED DESCRIPTION

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0051] The present invention is described in further detail below with reference to the accompanying drawings:

[0052] like Figure 1 As shown, a change assessment method for an application system according to the present invention includes:

[0053] When the problem management module finds a problem in the application system that cannot be fundamentally solved, it submits a change request to the change management module to make changes to the application system;

[0054] According to the characteristics of the application system, corresponding change assessment factors are introduced for different business types, and the object information corresponding to each change assessment factor is collected after the change operation is completed;

[0055] According to the change assessment factors, the TF value of each change assessment factor in all change assessment factors and the IDF value of the occurrence probability of each change assessment factor in all change assessment factors are determined based on the TF-IDF algorithm;

[0056] Obtain the object information corresponding to each change assessment factor before the change operation, and determine whether the change value of the object information corresponding to each change assessment factor before and after the change is within the preset valid range, and obtain the TH value of whether the change assessment factor is valid based on the determination result;

[0057] The sum of the products of the TF value, IDF value, and TH value of all change evaluation factors in the application system is calculated as the robustness score of the application system to evaluate the effectiveness of the application system after the change.

[0058] In this implementation, a change assessment factor is introduced into the changed application system, object information of the change assessment factors of different business types is collected, and a robustness score is obtained by analyzing and calculating the TF value, IDF value and TH value of the change assessment factor to evaluate whether the change of the application system is effective and whether the application system runs stably after the change, providing an effective reference basis for the next change.

[0059] Specifically, the goal of change management is to ensure that standard methods and procedures are used during the change implementation process, and that changes are implemented as quickly as possible to minimize the impact of business interruptions caused by the changes. Due to the wide variety of changes involved, here we focus on the change assessment of application software systems. After the application system change is completed, the quality of the change needs to be evaluated in a timely manner. At this time, it is necessary to establish a change assessment factor library. Combined with the usage characteristics of the business system, a post-change assessment model is established through the change factor assembly rules. By collecting the object information corresponding to the change factor, the status of the application system after the change is determined based on the assessment model. Whether the application system is running stably after the change is determined. Its main purpose is to measure the effectiveness of the change and provide an effective reference basis for the next change. At the same time, it is necessary to promptly obtain whether the application system is running stably after the change and whether it has any impact on the business.

[0060] Change management refers to the service management process for controlling changes to any aspect of the infrastructure or services in order to complete them with minimal disruption. When the problem management module discovers the root cause of a problem through investigation and analysis but cannot develop an appropriate solution to fundamentally resolve it, the problem management module needs to submit a change request to the change management module, thereby implementing the necessary changes to the application system to fundamentally eliminate the root cause of the problem. Generally, resolving the root cause of an application system problem requires a change to the application system. Once the change is completed, a timely and effective evaluation of the change is required to determine its effectiveness.

[0061] like Figure 2 As shown, in the above embodiment, preferably, change assessment factors are introduced according to the characteristics of the business system, specifically including CPU usage, memory usage, disk IO throughput, network bandwidth utilization, number of database connections, number of database operations per unit time, page response time, and interface response time. The specific business type definitions of the change assessment factors are as follows:

[0062] Change Assessment Factor type CPU usage Host Memory usage Host Disk IO throughput Host Network bandwidth utilization network Number of database connections database Number of database operations per unit time database Page response time application Interface response time application

[0063] In the above embodiment, preferably, the TF value of each change assessment factor in all change assessment factors is determined based on the TF-IDF (term frequency–inverse document frequency, TF, word frequency, IDF, inverse document frequency index) algorithm, and the specific process includes:

[0064] Determine the number of times W(P, T) a change assessment factor T is used to mark a service type P;

[0065] Calculate the ratio of the number of times the change assessment factor T is marked to the number of all change assessment factors of the current service type P as the TF value. For details, refer to the following formula:

[0066]

[0067] Where W(P, T) is the number of times the change assessment factor T is used to mark the service type P, ∑W(P, T i ) is the number of all change evaluation factors of the current business type P.

[0068] For example, the number of change assessment factors belonging to host 1 is 2, the number of change assessment factors belonging to host 2 is 2, the number of change assessment factors belonging to network 1 is 1, and the number of change assessment factors belonging to business performance is 2. Then the TF of the change assessment factor of host 1 is 2 / (2+1+2).

[0069] Determine the IDF value of each change assessment factor's probability of occurrence among all change assessment factors, which is used to indicate the scarcity of the change assessment factor T among all change assessment factors. If a change assessment factor T has a very low probability of occurrence and is also used to mark a certain business type, this will make the relationship between the business type and the change assessment factor T closer. The specific process includes:

[0070] Calculate the sum of all business types and all change assessment factors, and calculate the ratio of this sum to the sum of all types corresponding to the current change assessment factor as the IDF value. For details, refer to the following formula:

[0071]

[0072] Among them, ∑∑W(P i , T) is the sum of all business types and all change assessment factors, ∑W(P i , T i ) is the sum of all types corresponding to the current change assessment factor.

[0073] In the above embodiment, preferably, the object information corresponding to each change assessment factor before the change operation is obtained, and whether the change value of the object information corresponding to each change assessment factor before and after the change is determined to be within a preset valid range is determined, and the TH value of whether the change assessment factor is valid is obtained according to the determination result. The specific process includes:

[0074] Collect the object information of each change assessment factor in the application system before the change operation;

[0075] For each change assessment factor, the change value of the object information before and after the change operation is compared. If the change value exceeds the preset valid range, the current change assessment factor is determined to be invalid and the TH value is set to 0. If the change value does not exceed the preset valid range, the current change assessment factor is determined to be valid and the TH value is set to 1.

[0076] In the above embodiment, preferably, the sum of the products of the TF value, IDF value, and TH value of all change evaluation factors in the application system is calculated as the robustness score of the application system. The specific process of evaluating the effectiveness of the changed application system includes:

[0077] Calculate the product of TF value, IDF value and TH value of each change assessment factor;

[0078] Calculate the sum of the products of all change assessment factors in the application system as the robustness score of the application system;

[0079] The effectiveness of the application system change is evaluated based on the application system's robustness score. The lower the robustness score, the worse the effectiveness of the application system change operation.

[0080] Specifically, the calculation formula for the robustness score is:

[0081] Robustness score = (type 1*TF*IDF*TH+type 2*TF*IDF*TH+...+type n*TF*IDF*TH)*100%.

[0082] The change assessment method for the application system disclosed in the above-mentioned embodiment establishes a change assessment factor library, combines the usage characteristics of the business system, and establishes a post-change assessment model through change factor assembly rules. By collecting object information corresponding to the change assessment factors, the status of the application system after the change is obtained based on the change assessment model, and whether the application system runs stably after the change. Its main purpose is to measure the effectiveness of the change and provide an effective reference basis for the next change.

[0083] like Figure 3 As shown, the present invention further proposes an application system change assessment system, which applies the application system change assessment method disclosed in any of the above embodiments, including:

[0084] The change operation module 11 is used to perform a change operation on the application system when the problem management module finds that the application system has a problem that cannot be fundamentally solved and submits a change request to the change management module;

[0085] The factor collection module 12 is used to introduce corresponding change assessment factors for different business types according to the characteristics of the application system, and collect the object information corresponding to each change assessment factor after the change operation is completed;

[0086] The change analysis module 13 is used to determine the proportion TF value of each change assessment factor in all change assessment factors and the occurrence probability IDF value of each change assessment factor in all change assessment factors based on the TF-IDF algorithm according to the change assessment factors;

[0087] The change analysis module 13 is further configured to obtain the object information corresponding to each change assessment factor before the change operation, and determine whether the change value of the object information corresponding to each change assessment factor before and after the change is within a preset valid range, and obtain a TH value indicating whether the change assessment factor is valid based on the determination result;

[0088] The change evaluation module 14 is used to calculate the sum of the products of the TF value, IDF value and TH value of all change evaluation factors in the application system as the robustness score of the application system, and evaluate the effectiveness of the application system after the change.

[0089] In this implementation, a change assessment factor is introduced into the changed application system, object information of the change assessment factors of different business types is collected, and a robustness score is obtained by analyzing and calculating the TF value, IDF value and TH value of the change assessment factor to evaluate whether the change of the application system is effective and whether the application system runs stably after the change, providing an effective reference basis for the next change.

[0090] In the above embodiment, preferably, the change operation module 11 is specifically used to:

[0091] The change management module completes any change operation of the infrastructure or service within the preset interruption time according to the change request and the preset service management process.

[0092] In the above embodiment, preferably, the change analysis module 13 is specifically used to:

[0093] Determine the number of times w(P, T) a change assessment factor T is used to mark a service type P;

[0094] Calculate the ratio of the number of times the change assessment factor T is marked to the number of all change assessment factors of the current service type P as the TF value;

[0095] Calculate the sum of all business types and all change assessment factors, and calculate the ratio of this sum to the sum of all types corresponding to the current change assessment factor as the IDF value;

[0096] Collect the object information of each change assessment factor in the application system before the change operation;

[0097] For each change assessment factor, the change value of the object information before and after the change operation is compared. If the change value exceeds the preset valid range, the current change assessment factor is determined to be invalid and the TH value is set to 0. If the change value does not exceed the preset valid range, the current change assessment factor is determined to be valid and the TH value is set to 1.

[0098] In the above embodiment, preferably, the change assessment module 14 is specifically configured to:

[0099] Calculate the product of TF value, IDF value and TH value of each change assessment factor;

[0100] Calculate the sum of the products of all change assessment factors in the application system as the robustness score of the application system;

[0101] The effectiveness of the application system change is evaluated based on the application system's robustness score. The lower the robustness score, the worse the effectiveness of the application system change operation.

[0102] According to the change assessment system of the application system disclosed in the above embodiment, the functions to be implemented by each module thereof correspond to the steps of the change assessment method of the application system disclosed in the above embodiment. The specific implementation process can be found in the above embodiment and will not be repeated here.

[0103] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for evaluating changes in an application system, characterized in that: include: When the problem management module finds that an application system has a problem that cannot be fundamentally solved, it submits a change request to the change management module to perform a change operation on the application system; According to the characteristics of the application system, corresponding change assessment factors are introduced for different business types, and the object information corresponding to each change assessment factor is collected after the change operation is completed; According to the change assessment factors, determine the proportion TF value of each change assessment factor in all change assessment factors and the occurrence probability IDF value of each change assessment factor in all change assessment factors based on the TF-IDF algorithm; Obtain the object information corresponding to each change assessment factor before the change operation, and determine whether the change value of the object information corresponding to each change assessment factor before and after the change is within the preset valid range, and obtain the TH value of whether the change assessment factor is valid based on the determination result; The sum of the products of the TF value, IDF value and TH value of all change evaluation factors in the application system is calculated as the robustness score of the application system to evaluate the effectiveness of the application system after the change.

2. The change evaluation method for an application system according to claim 1, characterized in that: The specific process of submitting a change request to the change management module to perform a change operation on the application system includes: The change management module completes the change operation of any aspect of the infrastructure or service within the preset interruption time according to the change request and the preset service management process.

3. The change evaluation method for an application system according to claim 1, characterized in that: The change assessment factors include CPU usage, memory usage, disk IO throughput, network bandwidth utilization, number of database connections, number of database operations per unit time, page response time, and interface response time.

4. The change evaluation method for an application system according to claim 1, characterized in that: The specific process of determining the TF value of the proportion of each change assessment factor in all change assessment factors and the IDF value of the occurrence probability of each change assessment factor in all change assessment factors based on the TF-IDF algorithm includes: Determine the number of times w(P, T) a change assessment factor T is used to mark a service type P; Calculate the ratio of the number of times the change assessment factor T is marked to the number of all change assessment factors of the current service type P as the TF value; The sum of all business types and all change assessment factors is calculated, and the ratio of the sum to the sum of all types corresponding to the current change assessment factor is calculated as the IDF value.

5. The change evaluation method for an application system according to claim 4, characterized in that: The specific process of obtaining the object information corresponding to each change assessment factor before the change operation, determining whether the change value of the object information corresponding to each change assessment factor before and after the change is within a preset valid range, and obtaining the TH value of whether the change assessment factor is valid according to the determination result includes: Collecting object information corresponding to each change assessment factor of the application system before the change operation; For each change assessment factor, the change value of the object information before and after the change operation is compared. If the change value exceeds the preset valid range, the current change assessment factor is determined to be invalid and the TH value is set to 0. If the change value does not exceed the preset valid range, the current change assessment factor is determined to be valid and the TH value is set to 1.

6. The change evaluation method for an application system according to claim 5, characterized in that: The specific process of calculating the sum of the products of the TF value, IDF value and TH value of all change evaluation factors in the application system as the robustness score of the application system and evaluating the effectiveness of the application system after the change includes: Calculate the product of TF value, IDF value and TH value of each change assessment factor; Calculating the sum of the products of all change assessment factors in the application system as the robustness score of the application system; The effectiveness of the change of the application system is evaluated according to the robustness score of the application system. The lower the robustness score, the worse the effectiveness of the change operation of the application system.

7. A change assessment system for an application system, characterized in that: The change assessment method for an application system according to any one of claims 1 to 6 comprises: A change operation module is used to perform a change operation on the application system when the problem management module finds that the application system has a problem that cannot be fundamentally solved and submits a change request to the change management module; The factor collection module is used to introduce corresponding change assessment factors for different business types based on the characteristics of the application system, and collect the object information corresponding to each change assessment factor after the change operation is completed; A change analysis module is used to determine, based on the change assessment factors and the TF-IDF algorithm, the proportion TF value of each change assessment factor in all change assessment factors and the IDF value of the occurrence probability of each change assessment factor in all change assessment factors; The change analysis module is further configured to obtain object information corresponding to each change assessment factor before the change operation, and determine whether the change value of the object information corresponding to each change assessment factor before and after the change is within a preset valid range, and obtain a TH value indicating whether the change assessment factor is valid based on the determination result; The change evaluation module is used to calculate the sum of the products of the TF value, IDF value and TH value of all change evaluation factors in the application system as the robustness score of the application system, and evaluate the effectiveness of the application system after the change.

8. The application system change assessment system according to claim 7, characterized in that: The change operation module is specifically used for: The change management module completes the change operation of any aspect of the infrastructure or service within the preset interruption time according to the change request and the preset service management process.

9. The application system change assessment system according to claim 7, characterized in that: The change analysis module is specifically used for: Determine the number of times w(P, T) a change assessment factor T is used to mark a service type P; Calculate the ratio of the number of times the change assessment factor T is marked to the number of all change assessment factors of the current service type P as the TF value; Calculate the sum of all business types and all change assessment factors, and calculate the ratio of this sum to the sum of all types corresponding to the current change assessment factor as the IDF value; Collecting object information corresponding to each change assessment factor of the application system before the change operation; For each change assessment factor, the change value of the object information before and after the change operation is compared. If the change value exceeds the preset valid range, the current change assessment factor is determined to be invalid and the TH value is set to 0. If the change value does not exceed the preset valid range, the current change assessment factor is determined to be valid and the TH value is set to 1.

10. The application system change assessment system according to claim 9, characterized in that: The change assessment module is specifically used to: Calculate the product of TF value, IDF value and TH value of each change assessment factor; Calculating the sum of the products of all change assessment factors in the application system as the robustness score of the application system; The effectiveness of the change of the application system is evaluated according to the robustness score of the application system. The lower the robustness score, the worse the effectiveness of the change operation of the application system.

Images