Business system upgrade evaluation method, device, equipment, medium and product

Abstract

The invention discloses a business system upgrade evaluation method, device, equipment, medium and product, which are applied to the financial science and technology field, and comprise the following steps: collecting flow data, and determining a flow characteristic analysis result of a to-be-evaluated system according to the flow data; obtaining a test case, and determining a case feature analysis result of the to-be-evaluated system according to the test case and the flow data; and determining a system upgrading evaluation result according to the flow feature analysis result and the use case feature analysis result. Flow feature analysis, case feature analysis and coverage rate analysis results are visually displayed in a system upgrading evaluation result, so that a developer can determine potential problems and advantages of an upgraded new system in various aspects and know the performance of a to-be-evaluated system, a clear direction and basis are provided for optimization of the new system, and the development efficiency is improved. The performance of the upgraded system is not less than that of the old system, the operation safety and stability of the new system after the system is online are guaranteed, and the fault rate is reduced while the manual workload is reduced.

Description

Technical Field

[0001] The present invention relates to the field of financial technology, and in particular to a business system upgrade evaluation method, device, equipment, medium and product. Background Art

[0002] In recent years, architecture transformation has gradually become the second major factor in business system changes and upgrades, which has had a significant impact on the security and stability of online systems during the upgrade process. The development team relies more on the testing team to ensure the quality of system architecture upgrades through existing automated test cases and manual verification experience. Some systems are too old and have insufficient historical asset accumulation, so the integrity of quality control itself has been greatly challenged.

[0003] The quality control after the existing technical system upgrade adopts manual verification, which relies too much on offline feedback from testers and ignores the feedback from the system itself, weakening the development team's weak understanding of quality assurance. In addition, the verification after the upgrade focuses more on the functional dimension and pays insufficient attention to the system's performance, cloud-native capabilities, etc. Summary of the invention

[0004] The present invention provides a business system upgrade evaluation method, device, equipment, medium and product to determine the system upgrade evaluation result, thereby ensuring the operational security and stability of the new system after it goes online.

[0005] According to one aspect of the present invention, a method for evaluating a business system upgrade is provided, the method comprising:

[0006] Collect flow data and determine the flow characteristic analysis results of the system to be evaluated based on the flow data;

[0007] Obtain test cases, and determine the use case feature analysis results of the system to be evaluated based on the test cases and traffic data;

[0008] Determine the system upgrade evaluation results based on the traffic characteristics analysis results and use case characteristics analysis results.

[0009] Optionally, collecting traffic data includes: treating the business system before the upgrade as the old business system and the business system after the upgrade as the system to be evaluated; obtaining collection instructions, wherein the collection instructions include collection time and business type; and performing online traffic collection on the old business system based on the collection instructions to obtain traffic data.

[0010] Optionally, the traffic characteristic analysis results of the system to be evaluated are determined based on the traffic data, including: desensitizing the traffic data to generate desensitized traffic data; playing back the desensitized traffic data in the system to be evaluated to determine the traffic trend indicators of the system to be evaluated, and determining the traffic characteristic analysis results based on the traffic trend indicators, wherein the traffic trend indicators include traffic trends, average service time and database connection pool.

[0011] Optionally, the use case feature analysis results of the system to be evaluated are determined based on the test cases and traffic data, including: using an equivalent methodology to classify the traffic data to generate various online use cases; inputting online use cases and test cases into the system to be evaluated for testing to obtain use case comparison results; and determining the use case feature analysis results of the system to be evaluated based on the use case comparison results.

[0012] Optionally, the system upgrade evaluation results are determined based on the traffic feature analysis results and the use case feature analysis results, including: determining the current program coverage of the system to be evaluated based on the traffic feature analysis results and the use case feature analysis results, and obtaining the historical program coverage; determining the program coverage analysis results based on the current program coverage and the historical program coverage; and visualizing the traffic feature analysis results, the use case feature analysis results, and the program coverage analysis results to generate the system upgrade evaluation results.

[0013] Optionally, after obtaining the historical program coverage, the method further includes: when the current program coverage is less than the historical program coverage, obtaining a customized use case based on the current coverage; and performing a use case test on the system to be evaluated based on the customized use case.

[0014] According to another aspect of the present invention, a service system upgrade evaluation device is provided, the device comprising:

[0015] A flow characteristic analysis result determination module is used to collect flow data and determine the flow characteristic analysis result of the system to be evaluated based on the flow data;

[0016] A use case feature analysis result determination module is used to obtain test cases and determine the use case feature analysis results of the system to be evaluated based on the test cases and traffic data;

[0017] The system upgrade evaluation result determination module is used to determine the system upgrade evaluation result based on the traffic feature analysis results and the use case feature analysis results.

[0018] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0019] at least one processor;

[0020] and a memory communicatively coupled to the at least one processor;

[0021] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a business system upgrade evaluation method described in any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a business system upgrade evaluation method described in any embodiment of the present invention when executed.

[0023] According to another aspect of the present invention, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the computer program implements a business system upgrade evaluation method according to any embodiment of the present invention.

[0024] The technical solution of the embodiment of the present invention visualizes the traffic feature analysis, use case feature analysis and coverage analysis results in the system upgrade evaluation results, which helps developers to determine the potential problems and advantages of the upgraded new system in various aspects and understand the performance of the system to be evaluated, providing a clear direction and basis for the optimization of the new system, and can ensure that the performance of the system after the upgrade is no less than that of the old system, and ensure the operational safety and stability of the new system after it is put into operation, reducing the manual workload while also reducing the failure rate.

[0025] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 is a flow chart of a business system upgrade evaluation method provided according to Embodiment 1 of the present invention;

[0028] Figure 2 is a flowchart of another business system upgrade evaluation method provided according to Embodiment 2 of the present invention;

[0029] Figure 3 This is a schematic diagram of the structure of a business system upgrade evaluation device provided according to Embodiment 3 of the present invention;

[0030] Figure 4 It is a structural diagram of an electronic device for implementing a business system upgrade evaluation method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Embodiment 1

[0034] Figure 1 A flowchart of a business system upgrade evaluation method is provided for the first embodiment of the present invention. This embodiment is applicable to financial business system upgrade scenarios. The method can be performed by a business system upgrade evaluation device. The business system upgrade evaluation device can be implemented in the form of hardware and / or software. The business system upgrade evaluation device can be configured in a computer controller. Figure 1 As shown, the method includes:

[0035] S110, collecting flow data, and determining flow characteristic analysis results of the system to be evaluated based on the flow data.

[0036] The business system refers to the financial business system, which involves functions such as account management, fund transactions, and risk assessment. Traffic data refers to the data information related to data flow generated during the operation of the business system, including but not limited to the number of network requests generated when users access the system, the number of bytes of data transmitted, and the size of data traffic when different business function modules are accessed. Traffic data can reflect the system's busyness in different time periods, the frequency of use of each function module, etc., and is an important basis for analyzing the system's operating status. The system to be evaluated refers to the upgraded new business system.

[0037] Optionally, collecting traffic data includes: treating the business system before the upgrade as the old business system and the business system after the upgrade as the system to be evaluated; obtaining collection instructions, wherein the collection instructions include collection time and business type; and performing online traffic collection on the old business system based on the collection instructions to obtain traffic data.

[0038] It should be noted that when evaluating the effect of business system upgrades, the object of comparison needs to be clear. The controller will use the business system before the upgrade as the old business system, representing the original operating status and performance of the system before the upgrade, and the upgraded business system as the system to be evaluated. By comparing it with the old business system, it is determined whether the upgrade has achieved the expected goal. The controller refers to the computer controller that performs system upgrade evaluation.

[0039] The collection time specifies the time period during which traffic data is collected. By setting a specific collection time, you can obtain traffic data of the business system in a specific operating period in a targeted manner, and more accurately reflect the traffic characteristics of the business system during actual operation. Business systems usually cover a variety of different business types, and the traffic generated by different business types during execution is also different. After clarifying the business type, you can collect traffic data separately according to different business types, so as to have a more detailed understanding of the traffic consumption and other conditions corresponding to each business function.

[0040] Specifically, the controller can carry out online traffic collection work for the old business system according to the acquired collection instructions. Specifically, it is achieved by deploying relevant monitoring tools in the operating environment of the old business system or using the system's own logging functions. During the collection, according to the requirements of the collection instructions, within the specified collection time, for the specified business type, all information related to data flow will be recorded as traffic data, such as the size of the network data packet when the user initiates a request for a certain business function, the number of requests, the direction of data transmission, etc.

[0041] Optionally, the traffic characteristic analysis results of the system to be evaluated are determined based on the traffic data, including: desensitizing the traffic data to generate desensitized traffic data; playing back the desensitized traffic data in the system to be evaluated to determine the traffic trend indicators of the system to be evaluated, and determining the traffic characteristic analysis results based on the traffic trend indicators, wherein the traffic trend indicators include traffic trends, average service time and database connection pool.

[0042] It is known that, since it is a financial business system, some sensitive information may be included when processing traffic data, such as the user's personal identity, specific account balance, etc. In order to protect sensitive information from being leaked and to conduct effective analysis based on the data, data desensitization is required. Data desensitization is to deform the sensitive fields in the original traffic data through specific algorithms and rules, so that it no longer has the ability to identify personal identity or sensitive business information, but retains the basic structure and statistical characteristics of the data. For example, for the user's ID number field, it can be converted into a string of characters with no practical meaning but similar format through replacement, encryption, etc. For the account balance, fuzzy processing can be used to retain only the approximate amount range.

[0043] Specifically, the purpose of replaying the desensitized traffic data in the system to be evaluated is to simulate the real business traffic scenario and test the system to be evaluated in a situation close to the actual operation. The specific operation is to re-enter the desensitized traffic data into the system to be evaluated for playback according to a certain time sequence and business logic through special tools or the system's built-in test function.

[0044] It should be noted that during the data playback process, the traffic trend can be determined by continuously monitoring the processing of traffic by the system to be evaluated in different time periods. For example, observe whether the traffic entering the system every hour, day or week is gradually increasing, gradually decreasing or remaining relatively stable. The traffic trend can reflect the carrying capacity and adaptability of the system to be evaluated to traffic over time during the business development process. It is an important indicator for judging whether the system performance is stable and whether it can meet the needs of business development. In addition, for each service provided by the system, the time spent on each service execution from start to end can be recorded to determine the average service time. The average service time can reflect the response speed of each service when the system to be evaluated processes business traffic. If the average service time is long, it may mean that the system has a performance bottleneck and needs further optimization. Otherwise, it means that the system performs well in terms of response speed. The database connection pool is a mechanism for managing database connections. It creates a certain number of database connections in advance, and obtains connections from the connection pool when the system needs to access the database, and returns them to the connection pool after use. By observing the usage of the database connection pool during playback, such as whether the connections in the connection pool are often busy and whether there are enough connections available, we can understand the efficiency of the database resource utilization of the system to be evaluated and whether there is a database resource bottleneck. If the connection pool is often insufficient, it may cause slow service response or even failure.

[0045] In a specific implementation, when analyzing traffic characteristics, the online traffic of the old business system can be continuously collected to analyze the daily traffic trend, the performance of the system under different traffic conditions, the system performance under scenarios where upstream and downstream systems influence each other, etc., to help developers consider high-performance feature requirements, the implementation of self-protection mechanisms under the influence of external failures, and the formulation of emergency measures when building new systems.

[0046] S120: Acquire test cases, and determine the use case feature analysis results of the system to be evaluated based on the test cases and traffic data.

[0047] Among them, test cases refer to a collection of input data, operation steps and expected output results specially designed to test whether the business system meets specific requirements, whether its functions are normal, whether its performance is up to standard, etc., including operational test cases for normal processes and test cases for boundary conditions and abnormal situation handling. By executing test cases, the actual operation of the business system can be verified.

[0048] Optionally, the use case feature analysis results of the system to be evaluated are determined based on the test cases and traffic data, including: using an equivalent methodology to classify the traffic data to generate various online use cases; inputting online use cases and test cases into the system to be evaluated for testing to obtain use case comparison results; and determining the use case feature analysis results of the system to be evaluated based on the use case comparison results.

[0049] Among them, equivalence class division refers to dividing the set of input data into several subsets, namely equivalence classes, according to certain rules. In this embodiment, the massive traffic data is reasonably divided according to certain characteristics of the traffic data, such as the size range of the traffic, the business type that generates the traffic, the direction of the traffic, etc. For example, traffic with similar size and the same business type are classified into one category. Then, different types of subsets are sorted out from the complex and diverse traffic data, and each subset is used as an online use case.

[0050] Specifically, the controller will input both online use cases and test cases into the system to be evaluated for testing, and the system will process each use case according to its own business logic and functional modules. For online use cases, the system will simulate and process various situations in real business traffic scenarios. For test cases, the system will execute its preset operation steps and determine whether the expected output results can be achieved. Finally, the two will be compared to generate use case comparison results, and the use case comparison results will be used as use case feature analysis results. For example, record the success rate of online use cases and test cases in the system, that is, the proportion of use cases that can be successfully completed and achieve the expected results; count the system resources consumed during the execution of online use cases and test cases, such as CPU usage, memory usage, etc.; observe the system response time when online use cases and test cases are executed, that is, the time it takes from the input of the use case into the system to the system giving the processing results, etc.

[0051] Furthermore, based on the results of the use case feature analysis, the controller can feed back and analyze the effectiveness and completeness of the test cases in the test environment, and confirm the consistency and stock capacity of the new system implementation capability with the old system. For example, if the use case comparison results show that the execution success rates of online use cases and test cases are both very high, and there is little difference between the two in terms of system resource consumption and response time, then it can be concluded that the system to be evaluated has good stability and consistency in processing various use cases. If the use case comparison results show that the execution success rate of online use cases is low, or there are large differences between online use cases and test cases in terms of system resource consumption, response time, etc., then it can be judged that the system to be evaluated may have problems in processing different types of use cases, such as insufficient use case processing capabilities in real traffic scenarios, or performance optimization needs to be improved. At this time, developers can adjust the new system based on the use case feature analysis results to ensure the stability of the new system after it goes online.

[0052] S130. Determine a system upgrade evaluation result based on the traffic feature analysis result and the use case feature analysis result.

[0053] Among them, the system upgrade evaluation results are visual evaluation results after the business system is upgraded. For example, if the traffic feature analysis results show that the overall system traffic is relatively stable and has a certain margin, and the use case feature analysis results show that the new test case execution success rate is high and the execution time is short, then it can be preliminarily judged that the system may have better performance and functions after the upgrade. If there are multiple indicators in the traffic feature analysis results and the use case feature analysis results that are not ideal, such as frequent traffic congestion and a high test case execution failure rate, then it may indicate that there are risks in the system upgrade or that further optimization and adjustment are needed before considering the upgrade.

[0054] The technical solution of the embodiment of the present invention visualizes the traffic feature analysis, use case feature analysis and coverage analysis results in the system upgrade evaluation results, which helps developers to determine the potential problems and advantages of the upgraded new system in various aspects and understand the performance of the system to be evaluated, providing a clear direction and basis for the optimization of the new system, and can ensure that the performance of the system after the upgrade is no less than that of the old system, and ensure the operational safety and stability of the new system after it is put into operation, reducing the manual workload while also reducing the failure rate.

[0055] Embodiment 2

[0056] Figure 2 This is a flowchart of a business system upgrade evaluation method provided in Embodiment 2 of the present invention. This embodiment adds a specific process of determining the system upgrade evaluation result based on the traffic feature analysis results and the use case feature analysis results on the basis of Embodiment 1. Among them, the specific contents of steps S210-S220 are roughly the same as steps S110-S120 in Embodiment 1, so they will not be repeated in this embodiment. Figure 2 As shown, the method includes:

[0057] S210: Collect flow data, and determine flow characteristic analysis results of the system to be evaluated based on the flow data.

[0058] Optionally, collecting traffic data includes: treating the business system before the upgrade as the old business system and the business system after the upgrade as the system to be evaluated; obtaining collection instructions, wherein the collection instructions include collection time and business type; and performing online traffic collection on the old business system based on the collection instructions to obtain traffic data.

[0059] Optionally, the traffic characteristic analysis results of the system to be evaluated are determined based on the traffic data, including: desensitizing the traffic data to generate desensitized traffic data; playing back the desensitized traffic data in the system to be evaluated to determine the traffic trend indicators of the system to be evaluated, and determining the traffic characteristic analysis results based on the traffic trend indicators, wherein the traffic trend indicators include traffic trends, average service time and database connection pool.

[0060] S220: Obtain test cases, and determine the use case feature analysis results of the system to be evaluated based on the test cases and traffic data.

[0061] Optionally, the use case feature analysis results of the system to be evaluated are determined based on the test cases and traffic data, including: using an equivalent methodology to classify the traffic data to generate various online use cases; inputting online use cases and test cases into the system to be evaluated for testing to obtain use case comparison results; and determining the use case feature analysis results of the system to be evaluated based on the use case comparison results.

[0062] S230. Determine the current program coverage of the system to be evaluated according to the traffic feature analysis results and the use case feature analysis results, and obtain the historical program coverage.

[0063] Among them, the traffic feature analysis results reflect the processing of traffic by the system to be evaluated during actual operation, such as traffic trends, average service time, database connection pool and other aspects. The use case feature analysis results show the execution success rate, resource consumption, response time and other conditions of the system when processing various test cases, including online use cases generated based on traffic data and specially designed test cases. By analyzing the traffic features and use case features, it is possible to determine which program modules in the system are frequently called during actual operation and testing, and which are relatively rarely involved. For example, if the amount of traffic data corresponding to a certain business function is large and the related test cases are frequently executed, then the program module that implements the business function is likely to be fully covered. Comprehensive analysis of traffic features and use case features can be used to obtain the current program coverage of the system to be evaluated, that is, the proportion of system programs that are covered by execution under the current actual operation and test conditions. The historical program coverage refers to the proportion of the system programs that were covered by execution before the upgrade. Specifically, the historical program coverage information can be obtained through the operation data, test reports and other materials previously recorded by the system.

[0064] S240: Determine a program coverage analysis result according to the current program coverage and the historical program coverage.

[0065] Specifically, the controller will compare the current program coverage with the historical program coverage. If the current program coverage is higher than the historical program coverage, it means that after the system is upgraded, more program modules are fully utilized. This may be due to the addition of new functions, optimization of business processes, etc., which makes the system's program execution coverage wider. If the current program coverage is lower than the historical program coverage, it means that the new system has problems in some aspects. For example, some program modules that were originally frequently called are no longer fully utilized. This may be due to functional adjustments, changes in business logic, etc., which cause some program modules to be idle. By comparing the two and considering the differences, the controller can determine the results of the program coverage analysis, such as judging whether the program coverage is expanding, shrinking, or maintaining stability.

[0066] Optionally, after obtaining the historical program coverage, the method further includes: when the current program coverage is less than the historical program coverage, obtaining a customized use case based on the current coverage; and performing a use case test on the system to be evaluated based on the customized use case.

[0067] Among them, when it is found that the current program coverage is less than the historical program coverage, it means that the proportion of the new system program being executed and covered is lower than that of the old system. At this time, the controller can specifically find out the use cases related to the program modules that are not fully covered in the current operation and testing. For example, if it is found that a specific business function is less involved in the current traffic data and the execution of conventional test cases, then a customized use case can be designed around the business function and its related program modules. Customized use cases are designed to specifically target those parts that are not fully covered to more comprehensively detect the operation of the system and ensure that each program module of the system can be effectively tested.

[0068] S250, visualize the traffic feature analysis results, the use case feature analysis results, and the program coverage analysis results to generate a system upgrade evaluation result.

[0069] Among them, visualization processing is to present complex data and analysis results in the form of intuitive graphics, charts, etc. Developers can further adjust the new system based on the system upgrade evaluation results. If the traffic trend is stable, the use case execution success rate is high, the program coverage rate is expanded, and the visualization shows a relatively good trend in all indicators, it can be determined that the system upgrade is successful and the system performance and functions are improved.

[0070] Furthermore, if the visualization results show that multiple indicators are not good, such as large traffic fluctuations, low use case execution success rate, reduced program coverage, etc., then it may be judged that there are problems with the system upgrade, and the system upgrade evaluation results need to be further optimized or re-evaluated. For example, the traffic feature analysis results show that some business functions have almost no traffic access. At this time, developers can consider whether to redesign or directly remove these functions based on the upgrade goals. For another example, the use case feature analysis results show that the settlement use cases of a financial system often fail after the upgrade. At this time, developers need to check whether the relevant calculation logic or data interface has been modified during the system upgrade process, and repair the erroneous code parts in a timely manner.

[0071] The technical solution of the embodiment of the present invention can intuitively compare the current and historical program coverage through the program coverage analysis results to understand the development and change trend of the system. The system upgrade evaluation results in an intuitive visual form facilitate developers to quickly understand and grasp the overall status of the new system, more clearly show the advantages and disadvantages of the new system, and provide strong support for decision-making.

[0072] Embodiment 3

[0073] Figure 3 This is a schematic diagram of the structure of a business system upgrade evaluation device provided by Embodiment 3 of the present invention. Figure 3 As shown, the device includes: a flow characteristic analysis result determination module 310, which is used to collect flow data and determine the flow characteristic analysis result of the system to be evaluated according to the flow data;

[0074] A use case feature analysis result determination module 320 is used to obtain a test case and determine a use case feature analysis result of the system to be evaluated based on the test case and traffic data;

[0075] The system upgrade evaluation result determination module 330 is used to determine the system upgrade evaluation result according to the traffic feature analysis result and the use case feature analysis result.

[0076] Optionally, the traffic feature analysis result determination module 310 specifically includes: a traffic data collection unit, used to: treat the business system before the upgrade as the old business system, and the business system after the upgrade as the system to be evaluated; obtain collection instructions, wherein the collection instructions include the collection time and business type; based on the collection instructions, perform online traffic collection on the old business system to obtain traffic data.

[0077] Optionally, the traffic feature analysis result determination module 310 specifically includes: a traffic feature analysis result determination unit, which is used to: desensitize the traffic data to generate desensitized traffic data; replay the desensitized traffic data in the system to be evaluated to determine the traffic trend indicators of the system to be evaluated, and determine the traffic feature analysis results based on the traffic trend indicators, wherein the traffic trend indicators include traffic trends, average service time and database connection pool.

[0078] Optionally, the use case feature analysis result determination module 320 is specifically used to: classify the traffic data using an equivalent methodology to generate various online use cases; input online use cases and test cases into the system to be evaluated for testing to obtain use case comparison results; and determine the use case feature analysis results of the system to be evaluated based on the use case comparison results.

[0079] Optionally, the system upgrade evaluation result determination module 330 is specifically used to: determine the current program coverage of the system to be evaluated based on the traffic feature analysis results and the use case feature analysis results, and obtain the historical program coverage; determine the program coverage analysis results based on the current program coverage and the historical program coverage; and visualize the traffic feature analysis results, the use case feature analysis results, and the program coverage analysis results to generate a system upgrade evaluation result.

[0080] Optionally, the device also includes: a customized use case acquisition module, which is used to acquire customized use cases based on current coverage after acquiring historical program coverage, when the current program coverage is less than the historical program coverage; and perform use case testing on the system to be evaluated based on the customized use cases.

[0081] The technical solution of the embodiment of the present invention visualizes the traffic feature analysis, use case feature analysis and coverage analysis results in the system upgrade evaluation results, which helps developers to determine the potential problems and advantages of the upgraded new system in various aspects and understand the performance of the system to be evaluated, providing a clear direction and basis for the optimization of the new system, and can ensure that the performance of the system after the upgrade is no less than that of the old system, and ensure the operational safety and stability of the new system after it is put into operation, reducing the manual workload while also reducing the failure rate.

[0082] A business system upgrade evaluation device provided in an embodiment of the present invention can execute a business system upgrade evaluation method provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0083] Embodiment 4

[0084] Figure 4A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0085] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0086] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0087] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a business system upgrade evaluation method.

[0088] In some embodiments, a business system upgrade evaluation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the business system upgrade evaluation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute a business system upgrade evaluation method in any other appropriate manner (e.g., by means of firmware).

[0089] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0090] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0091] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0092] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0093] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0094] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0095] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0096] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A business system upgrade evaluation method, characterized in that: include: Collecting flow data, and determining flow characteristic analysis results of the system to be evaluated based on the flow data; Obtaining a test case, and determining a test case feature analysis result of the system to be evaluated according to the test case and the traffic data; A system upgrade evaluation result is determined based on the traffic feature analysis result and the use case feature analysis result.

2. The method according to claim 1, characterized in that The collecting of flow data includes: The business system before the upgrade is regarded as the old business system, and the business system after the upgrade is regarded as the system to be evaluated; Obtaining a collection instruction, wherein the collection instruction includes a collection time and a service type; Based on the collection instruction, online traffic collection is performed on the old business system to obtain traffic data.

3. The method according to claim 2, characterized in that Determining the flow characteristic analysis result of the system to be evaluated according to the flow data includes: Desensitizing the traffic data to generate desensitized traffic data; The desensitized traffic data is replayed in the system to be evaluated to determine the traffic trend indicators of the system to be evaluated, and the traffic feature analysis results are determined according to the traffic trend indicators, wherein the traffic trend indicators include traffic trends, average service time and database connection pool.

4. The method according to claim 2, characterized in that: The determining the use case feature analysis result of the system to be evaluated according to the test case and the traffic data includes: Using equivalence methodology to classify the traffic data to generate various online use cases; Input the online use case and the test case into the system to be evaluated for testing to obtain a use case comparison result; The use case feature analysis results of the system to be evaluated are determined according to the use case comparison results.

5. The method according to claim 1, characterized in that The determining of the system upgrade evaluation result according to the traffic feature analysis result and the use case feature analysis result includes: Determine the current program coverage of the system to be evaluated according to the traffic feature analysis result and the use case feature analysis result, and obtain the historical program coverage; Determine a program coverage analysis result according to the current program coverage and the historical program coverage; The traffic feature analysis results, the use case feature analysis results and the program coverage analysis results are visualized to generate a system upgrade evaluation result.

6. The method according to claim 5, characterized in that After obtaining the historical program coverage, the method further includes: When the current program coverage is less than the historical program coverage, obtaining a customized use case based on the current coverage; Perform a use case test on the system to be evaluated based on the customized use case.

7. A business system upgrade evaluation device, characterized in that: include: A flow characteristic analysis result determination module is used to collect flow data and determine the flow characteristic analysis result of the system to be evaluated according to the flow data; A use case feature analysis result determination module is used to obtain a test case and determine a use case feature analysis result of the system to be evaluated based on the test case and the traffic data; The system upgrade evaluation result determination module is used to determine the system upgrade evaluation result according to the traffic feature analysis result and the use case feature analysis result.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 6.

9. A computer storage medium, characterized in that The computer storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images