A service system upgrade evaluation method, device, equipment, medium and product

Abstract

The application discloses a kind of service system upgrade evaluation method, device, equipment, medium and product, applied to the field of financial technology, comprising: collecting traffic data, determining the traffic characteristic analysis result of the system to be evaluated according to traffic data;Obtain test cases, determine the case characteristic analysis result of the system to be evaluated according to test cases and traffic data;Determine the system upgrade evaluation result according to traffic characteristic analysis result and case characteristic analysis result.The system upgrade evaluation result visualizes traffic characteristic analysis, case characteristic analysis and coverage analysis result, helps developers to determine the potential problems and advantages of the new system after upgrading in all aspects, understand the performance of the system to be evaluated, provides a clear direction and basis for the optimization of the new system, can guarantee the performance of the system after upgrading is not inferior to the old system, guarantee the operation safety and stability of the new system after on-line, reduce the workload while also reduce the failure rate.

Description

Technical Field

[0001] This invention relates to the field of financial technology, and in particular to a method, apparatus, equipment, medium, and product for evaluating business system upgrades. Background Technology

[0002] In recent years, architectural transformation has gradually become the second largest factor driving changes and upgrades to business systems, significantly impacting the security and stability of online systems during the upgrade process. Development teams increasingly rely on testing teams to ensure the quality of system architecture upgrades through existing automated test cases and manual verification experience. However, some systems are too outdated, lacking sufficient historical assets, which poses a significant challenge to the completeness of quality control.

[0003] The quality control of the upgraded existing technology system is carried out through manual verification, which relies too much on offline feedback from testers and ignores the feedback from the system itself. This weakens the development team's understanding of quality assurance. Furthermore, the upgraded verification focuses more on functional dimensions and pays insufficient attention to system performance and cloud-native capabilities. Summary of the Invention

[0004] This invention provides a method, apparatus, equipment, medium, and product for evaluating business system upgrades, in order to determine the evaluation results of system upgrades, thereby ensuring the operational security and stability of the new system after it goes live.

[0005] According to one aspect of the present invention, a method for evaluating the upgrade of a business system is provided, the method comprising:

[0006] Collect traffic data and determine the traffic characteristic analysis results of the system to be evaluated based on the traffic data;

[0007] Obtain test cases, and determine the test case feature analysis results of the system to be evaluated based on the test cases and traffic data;

[0008] The system upgrade evaluation results are determined based on the traffic characteristic analysis results and the use case characteristic analysis results.

[0009] Optionally, traffic data collection includes: treating the pre-upgrade business system as the old business system and the upgraded business system as the system to be evaluated; obtaining collection instructions, wherein the collection instructions include collection time and business type; and collecting online traffic from the old business system based on the collection instructions to obtain traffic data.

[0010] Optionally, the traffic characteristic analysis results of the system to be evaluated are determined based on the traffic data, including: data anonymization of the traffic data to generate anonymized traffic data; replaying the anonymized traffic data on the system to be evaluated to determine the traffic trend indicators of the system to be evaluated; and determining the traffic characteristic analysis results based on the traffic trend indicators, wherein the traffic trend indicators include traffic trend, average service time, and database connection pool.

[0011] Optionally, the use case feature analysis results of the system to be evaluated are determined based on the test cases and traffic data, including: classifying the traffic data using the equivalence methodology to generate online use cases; inputting the online use cases and test cases into the system to be evaluated for testing to obtain use case comparison results; and determining the use case feature analysis results of the system to be evaluated based on the use case comparison results.

[0012] Optionally, the system upgrade evaluation results are determined based on the traffic feature analysis results and the use case feature analysis results, including: determining the current program coverage of the system to be evaluated based on the traffic feature analysis results and the use case feature analysis results, and obtaining the historical program coverage; determining the program coverage analysis results based on the current program coverage and the historical program coverage; and visualizing the traffic feature analysis results, the use case feature analysis results, and the program coverage analysis results to generate the system upgrade evaluation results.

[0013] Optionally, after obtaining historical program coverage, the method further includes: when the current program coverage is less than the historical program coverage, obtaining customized test cases based on the current coverage; and performing test case testing on the system to be evaluated based on the customized test cases.

[0014] According to another aspect of the present invention, a business system upgrade evaluation apparatus is provided, the apparatus comprising:

[0015] The traffic characteristic analysis result determination module is used to collect traffic data and determine the traffic characteristic analysis result of the system to be evaluated based on the traffic data.

[0016] The test case feature analysis result determination module is used to obtain test cases and determine the test case feature analysis results of the system to be evaluated based on the test cases and traffic data.

[0017] The system upgrade evaluation result determination module is used to determine the system upgrade evaluation result based on the traffic characteristic analysis result and the use case characteristic analysis result.

[0018] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0019] At least one processor;

[0020] and a memory communicatively connected to the at least one processor;

[0021] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute a business system upgrade evaluation method according to any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement a business system upgrade evaluation method according to any embodiment of the present invention.

[0023] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements a business system upgrade evaluation method according to any embodiment of the present invention.

[0024] The technical solution of this invention visualizes the results of traffic feature analysis, use case feature analysis, and coverage analysis in the system upgrade evaluation results. This helps developers identify potential problems and advantages of the upgraded system in various aspects, understand the performance of the system to be evaluated, and provides a clear direction and basis for the optimization of the new system. It can ensure that the performance of the upgraded system is no less than that of the old system, ensure the operational security and stability of the new system after it goes online, and reduce the workload of manual labor while also reducing the failure rate.

[0025] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart of a business system upgrade evaluation method provided in Embodiment 1 of the present invention;

[0028] Figure 2 This is a flowchart of another business system upgrade evaluation method provided in Embodiment 2 of the present invention;

[0029] Figure 3 This is a schematic diagram of the structure of a business system upgrade evaluation device provided in Embodiment 3 of the present invention;

[0030] Figure 4 This is a schematic diagram of the structure of an electronic device that implements a business system upgrade evaluation method according to an embodiment of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] Example 1

[0034] Figure 1 This is a flowchart illustrating a business system upgrade evaluation method provided in Embodiment 1 of the present invention. This embodiment is applicable to financial business system upgrade scenarios. The method can be executed by a business system upgrade evaluation device, which can be implemented in hardware and / or software and can be configured in a computer controller. Figure 1 As shown, the method includes:

[0035] S110. Collect traffic data and determine the traffic characteristic analysis results of the system to be evaluated based on the traffic data.

[0036] The business system refers to the financial business system, which involves functions such as account management, fund transactions, and risk assessment. Traffic data refers to data related to data flow generated during the operation of the business system, including but not limited to the number of network requests generated when users access the system, the number of bytes transmitted, and the data traffic volume when different business function modules are accessed. Traffic data can reflect the system's workload at different times and the usage frequency of each function module, serving as an important basis for analyzing the system's operational status. The system to be evaluated refers to the upgraded new business system.

[0037] Optionally, traffic data collection includes: treating the pre-upgrade business system as the old business system and the upgraded business system as the system to be evaluated; obtaining collection instructions, wherein the collection instructions include collection time and business type; and collecting online traffic from the old business system based on the collection instructions to obtain traffic data.

[0038] It's important to note that when evaluating the effectiveness of a business system upgrade, the comparison object needs to be clearly defined. The controller uses the pre-upgrade business system as the old system, representing its original operating state and performance before the upgrade. The upgraded business system is used as the system to be evaluated. By comparing it with the old system, it's determined whether the upgrade achieved its intended goals. The controller refers to the computer controller that performs the system upgrade evaluation.

[0039] The collection time specifies the time period during which traffic data is collected. By setting a specific collection time, traffic data from the business system during a particular runtime period can be obtained more accurately, reflecting the actual traffic characteristics of the business system during operation. Business systems typically encompass multiple different business types, and the traffic generated during the execution of different business types varies. Once the business type is identified, traffic data can be collected separately for each business type to gain a more detailed understanding of the traffic consumption corresponding to each business function.

[0040] Specifically, the controller can collect online traffic from legacy business systems based on the received collection instructions. This is achieved by deploying relevant monitoring tools within the legacy business system's operating environment or by utilizing the system's built-in logging functions. During collection, the controller will record all information related to data flow for the specified business type within the defined collection time, according to the collection instructions. This includes information such as the size of network packets, the number of requests, and the direction of data transmission when a user initiates a request for a particular business function, as traffic data.

[0041] Optionally, the traffic characteristic analysis results of the system to be evaluated are determined based on the traffic data, including: data anonymization of the traffic data to generate anonymized traffic data; replaying the anonymized traffic data on the system to be evaluated to determine the traffic trend indicators of the system to be evaluated; and determining the traffic characteristic analysis results based on the traffic trend indicators, wherein the traffic trend indicators include traffic trend, average service time, and database connection pool.

[0042] As this is a financial business system, the traffic data it processes may contain sensitive information, such as users' personal identification details and specific account balances. To protect this sensitive information from leakage while still enabling effective data analysis, data anonymization is necessary. Data anonymization uses specific algorithms and rules to transform sensitive fields in the original traffic data, rendering them unable to identify individuals or sensitive business information, while retaining the data's basic structure and statistical characteristics. For example, a user's ID number can be transformed into a string of meaningless but similarly formatted characters through replacement or encryption; account balances can be obfuscated, retaining only a general range of amounts.

[0043] Specifically, replaying the anonymized traffic data in the system under evaluation is to simulate real-world business traffic scenarios, allowing the system to be tested under conditions close to actual operation. The specific operation involves using specialized tools or the system's built-in testing functions to re-enter the anonymized traffic data into the system under evaluation according to a specific time sequence and business logic.

[0044] It's important to note that during data playback, continuously monitoring the traffic handling of the system under evaluation at different time periods allows us to determine traffic trends. For example, we can observe whether the traffic entering the system gradually increases, decreases, or remains relatively stable within an hour, day, or week. Traffic trends reflect the system's capacity to handle and adapt to traffic over time during business operations, serving as a crucial indicator of system performance stability and its ability to meet business development needs. Furthermore, recording the time spent on each service execution from start to finish for each service provided by the system determines the average service latency. Average service latency reflects the response speed of each service in the system when handling business traffic. A longer average service latency may indicate a performance bottleneck requiring further optimization, while a shorter average latency suggests good system response speed. A database connection pool is a mechanism for managing database connections. It pre-creates a certain number of database connections and retrieves them from the pool when the system needs to access the database, returning them to the pool after use. By observing the usage of the database connection pool during replay, such as whether the connections in the pool are frequently busy or whether there are enough connections available, we can understand the efficiency of the system under evaluation in utilizing database resources and whether there are any database resource bottlenecks. If the connection pool is frequently insufficient, it may lead to slow service response or even failure.

[0045] In one specific implementation, during traffic characteristic analysis, continuous collection of online traffic from the old business system can be used to analyze daily traffic trends, system performance under different traffic conditions, and system performance in scenarios where upstream and downstream systems interact. This helps developers consider high-performance requirements, the implementation of self-protection mechanisms under the influence of external faults, and the formulation of emergency measures when building new systems.

[0046] S120. Obtain test cases and determine the test case feature analysis results of the system to be evaluated based on the test cases and traffic data.

[0047] Test cases refer to a set of input data, operation steps and expected output results specifically designed to test whether a business system meets specific requirements, functions normally, and performs up to standard. They include test cases for normal operation processes and test cases for handling boundary conditions and abnormal situations. By executing test cases, the actual operation of the business system can be verified.

[0048] Optionally, the use case feature analysis results of the system to be evaluated are determined based on the test cases and traffic data, including: classifying the traffic data using the equivalence methodology to generate online use cases; inputting the online use cases and test cases into the system to be evaluated for testing to obtain use case comparison results; and determining the use case feature analysis results of the system to be evaluated based on the use case comparison results.

[0049] Equivalence class partitioning refers to dividing a set of input data into several subsets, i.e., equivalence classes, according to certain characteristics of the traffic data, such as the range of traffic volume, the type of business generating the traffic, and the direction of traffic flow, to rationally divide the massive traffic data. For example, traffic volumes with similar sizes and the same business type are grouped into one class. Furthermore, different types of subsets are extracted from the complex and diverse traffic data, and each subset is treated as an online use case.

[0050] Specifically, the controller inputs both online test cases and test cases into the system to be evaluated for testing. The system processes each test case according to its own business logic and functional modules. For online test cases, the system simulates various situations under real business traffic scenarios. For test cases, the system executes its preset operation steps and determines whether the expected output result can be achieved. Finally, the two are compared to generate test case comparison results, which are used as test case feature analysis results. For example, the success rate of online and test cases in the system is recorded, that is, the proportion of test cases that can be successfully completed and achieve the expected results; the system resources consumed by online and test cases during execution, such as CPU utilization and memory usage, are statistically analyzed; and the system response time during the execution of online and test cases, that is, the time spent from the input of the test case into the system to the output of the processing result, etc.

[0051] Furthermore, based on the test case feature analysis results, the controller can provide feedback to analyze the effectiveness and completeness of test cases in the testing environment, confirming the consistency and existing capabilities of the new system with the old system. For example, if the test case comparison results show that the execution success rates of both online and test cases are high, and there is little difference between them in terms of system resource consumption and response time, then the test case feature analysis results indicate that the system under evaluation has good stability and consistency in handling various types of test cases. However, if the test case comparison results show that the execution success rate of online test cases is low, or there are significant differences between online and test cases in terms of system resource consumption and response time, then it can be determined that the system under evaluation may have problems in handling different types of test cases, such as insufficient ability to handle test cases in real traffic scenarios, or performance optimization needs improvement. In this case, developers can adjust the new system based on the test case feature analysis results to ensure the stability of the new system after its launch.

[0052] S130. Determine the system upgrade evaluation results based on the traffic characteristic analysis results and the use case characteristic analysis results.

[0053] The system upgrade evaluation results are visualized assessments of the upgraded business system. For example, if traffic characteristic analysis shows relatively stable overall system traffic with some margin, and test case characteristic analysis shows high success rates and short execution times for new test cases, it can be preliminarily judged that the upgraded system may perform well in terms of performance and functionality. However, if multiple indicators in the traffic characteristic analysis and test case characteristic analysis results are unsatisfactory, such as frequent traffic congestion and high test case failure rates, it may indicate risks associated with the system upgrade or the need for further optimization and adjustments before considering an upgrade.

[0054] The technical solution of this invention visualizes the results of traffic feature analysis, use case feature analysis, and coverage analysis in the system upgrade evaluation results. This helps developers identify potential problems and advantages of the upgraded system in various aspects, understand the performance of the system to be evaluated, and provides a clear direction and basis for the optimization of the new system. It can ensure that the performance of the upgraded system is no less than that of the old system, ensure the operational security and stability of the new system after it goes online, and reduce the workload of manual labor while also reducing the failure rate.

[0055] Example 2

[0056] Figure 2 This is a flowchart of a business system upgrade evaluation method provided in Embodiment 2 of the present invention. This embodiment adds a specific process for determining the system upgrade evaluation result based on traffic characteristic analysis results and use case characteristic analysis results, building upon Embodiment 1. The specific content of steps S210-S220 is largely the same as steps S110-S120 in Embodiment 1, and therefore will not be repeated in this embodiment. Figure 2 As shown, the method includes:

[0057] S210. Collect traffic data and determine the traffic characteristic analysis results of the system to be evaluated based on the traffic data.

[0058] Optionally, traffic data collection includes: treating the pre-upgrade business system as the old business system and the upgraded business system as the system to be evaluated; obtaining collection instructions, wherein the collection instructions include collection time and business type; and collecting online traffic from the old business system based on the collection instructions to obtain traffic data.

[0059] Optionally, the traffic characteristic analysis results of the system to be evaluated are determined based on the traffic data, including: data anonymization of the traffic data to generate anonymized traffic data; replaying the anonymized traffic data on the system to be evaluated to determine the traffic trend indicators of the system to be evaluated; and determining the traffic characteristic analysis results based on the traffic trend indicators, wherein the traffic trend indicators include traffic trend, average service time, and database connection pool.

[0060] S220. Obtain test cases and determine the test case feature analysis results of the system to be evaluated based on the test cases and traffic data.

[0061] Optionally, the use case feature analysis results of the system to be evaluated are determined based on the test cases and traffic data, including: classifying the traffic data using the equivalence methodology to generate online use cases; inputting the online use cases and test cases into the system to be evaluated for testing to obtain use case comparison results; and determining the use case feature analysis results of the system to be evaluated based on the use case comparison results.

[0062] S230. Determine the current program coverage of the system to be evaluated based on the traffic characteristic analysis results and the use case characteristic analysis results, and obtain the historical program coverage.

[0063] The traffic characteristic analysis results reflect the system's handling of traffic during actual operation, such as traffic trends, average service latency, and database connection pool performance. The use case characteristic analysis results demonstrate the system's success rate, resource consumption, and response time when processing various test cases, including online test cases generated from traffic data and specially designed test cases. By analyzing traffic and use case characteristics, it's possible to determine which program modules are frequently invoked during actual operation and testing, and which are relatively less frequently involved. For example, if a certain business function corresponds to a large volume of traffic data and related test cases are frequently executed, then the program module implementing that business function is likely to be fully covered. A comprehensive analysis of traffic and use case characteristics yields the current program coverage of the system under evaluation, i.e., the proportion of system programs executed and covered under current actual operation and testing conditions. Historical program coverage refers to the proportion of programs executed and covered before the system upgrade; this information can be obtained through previously recorded operational data, test reports, and other materials.

[0064] S240. Determine the program coverage analysis results based on the current program coverage and historical program coverage.

[0065] Specifically, the controller compares the current program coverage with historical program coverage. If the current program coverage is higher than the historical program coverage, it indicates that more program modules are being fully utilized after the system upgrade, possibly due to the addition of new features, optimization of business processes, or other reasons, resulting in a wider program execution coverage. Conversely, if the current program coverage is lower than the historical program coverage, it indicates that the new system has encountered problems in some aspects. For example, some program modules that were previously frequently called are no longer being fully utilized, possibly due to feature adjustments, changes in business logic, or other reasons leading to some program modules being idle. By comparing the two and considering the differences, the controller can determine the program coverage analysis results, such as whether the program coverage has expanded, decreased, or remained stable.

[0066] Optionally, after obtaining historical program coverage, the method further includes: when the current program coverage is less than the historical program coverage, obtaining customized test cases based on the current coverage; and performing test case testing on the system to be evaluated based on the customized test cases.

[0067] When the current program coverage is found to be lower than the historical program coverage, it indicates that the proportion of programs being executed and covered by the new system is lower than that of the old system. In this case, the controller can specifically identify use cases related to program modules that are not adequately covered in the current operation and testing. For example, if a specific business function is found to be less involved in current traffic data and the execution of regular test cases, then custom use cases can be designed around that business function and its related program modules. Custom use cases are designed specifically to target those insufficiently covered parts, to more comprehensively test the system's operation, and to ensure that all program modules of the system are effectively tested.

[0068] S250. Visualize the results of traffic feature analysis, use case feature analysis, and program coverage analysis to generate system upgrade evaluation results.

[0069] Visualization involves presenting complex data and analysis results in intuitive graphical and chart formats. Developers can then further adjust the new system based on the system upgrade evaluation results. If traffic trends are stable, use case execution success rates are high, program coverage is expanded, and visualizations show a generally positive trend across all indicators, the system upgrade can be considered successful, resulting in improved system performance and functionality.

[0070] Furthermore, if the visualization results show unfavorable performance across multiple metrics, such as large traffic fluctuations, low test case execution success rates, and reduced program coverage, it may indicate a problem with the system upgrade, requiring further optimization or a reassessment of the upgrade evaluation results. For example, if traffic characteristic analysis shows that some business functions have almost no traffic access, developers can consider whether to redesign or remove these functions, based on the upgrade goals. As another example, if test case characteristic analysis shows that a financial system's settlement test cases frequently malfunction after the upgrade, developers need to check whether related calculation logic or data interfaces were modified during the system upgrade process and promptly fix the erroneous code.

[0071] The technical solution of this invention allows for a direct comparison of current and historical program coverage through program coverage analysis, providing insight into system development and trends. Presenting system upgrade evaluation results in an intuitive and visual format facilitates developers' rapid understanding and grasp of the new system's overall status, clearly demonstrating its strengths and weaknesses and providing strong support for decision-making.

[0072] Example 3

[0073] Figure 3 This is a schematic diagram of a business system upgrade evaluation device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: a flow characteristic analysis result determination module 310, used to collect flow data and determine the flow characteristic analysis result of the system to be evaluated based on the flow data;

[0074] The use case feature analysis result determination module 320 is used to obtain test cases and determine the use case feature analysis results of the system to be evaluated based on the test cases and traffic data.

[0075] The system upgrade evaluation result determination module 330 is used to determine the system upgrade evaluation result based on the traffic characteristic analysis result and the use case characteristic analysis result.

[0076] Optionally, the traffic characteristic analysis result determination module 310 specifically includes: a traffic data acquisition unit, used to: take the business system before the upgrade as the old business system and the upgraded business system as the system to be evaluated; obtain acquisition instructions, wherein the acquisition instructions include acquisition time and business type; and perform online traffic acquisition on the old business system based on the acquisition instructions to obtain traffic data.

[0077] Optionally, the traffic feature analysis result determination module 310 specifically includes: a traffic feature analysis result determination unit, used for: performing data anonymization on traffic data to generate anonymized traffic data; replaying the anonymized traffic data in the system to be evaluated to determine the traffic trend indicators of the system to be evaluated; and determining the traffic feature analysis results based on the traffic trend indicators, wherein the traffic trend indicators include traffic trend, average service time, and database connection pool.

[0078] Optionally, the use case feature analysis result determination module 320 is specifically used for: classifying traffic data using equivalence methodology to generate various online use cases; inputting online use cases and test cases into the system to be evaluated for testing to obtain use case comparison results; and determining the use case feature analysis results of the system to be evaluated based on the use case comparison results.

[0079] Optionally, the system upgrade evaluation result determination module 330 is specifically used for: determining the current program coverage of the system to be evaluated based on the traffic feature analysis results and the use case feature analysis results, and obtaining the historical program coverage; determining the program coverage analysis results based on the current program coverage and the historical program coverage; and visualizing the traffic feature analysis results, the use case feature analysis results, and the program coverage analysis results to generate the system upgrade evaluation results.

[0080] Optionally, the device further includes: a customized test case acquisition module, used to acquire customized test cases based on the current coverage when the current coverage is less than the historical coverage after acquiring the historical program coverage; and to perform test case testing on the system to be evaluated based on the customized test cases.

[0081] The technical solution of this invention visualizes the results of traffic feature analysis, use case feature analysis, and coverage analysis in the system upgrade evaluation results. This helps developers identify potential problems and advantages of the upgraded system in various aspects, understand the performance of the system to be evaluated, and provides a clear direction and basis for the optimization of the new system. It can ensure that the performance of the upgraded system is no less than that of the old system, ensure the operational security and stability of the new system after it goes online, and reduce the workload of manual labor while also reducing the failure rate.

[0082] The business system upgrade evaluation device provided in this embodiment of the invention can execute a business system upgrade evaluation method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0083] Example 4

[0084] Figure 4A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0085] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0086] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0087] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a business system upgrade evaluation method.

[0088] In some embodiments, a business system upgrade evaluation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the business system upgrade evaluation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform a business system upgrade evaluation method by any other suitable means (e.g., by means of firmware).

[0089] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0090] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0091] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0092] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0093] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0094] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0095] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0096] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for evaluating business system upgrades, characterized in that, include: Collect traffic data and determine the traffic characteristic analysis results of the system to be evaluated based on the traffic data; Obtain test cases, and determine the test case feature analysis results of the system to be evaluated based on the test cases and the traffic data; The system upgrade evaluation result is determined based on the traffic characteristic analysis result and the use case characteristic analysis result; The step of determining the use case feature analysis results of the system to be evaluated based on the test cases and the traffic data includes: Based on the size range of the traffic data, the type of business that generates the traffic data, and the direction of the traffic data flow, the traffic data is classified using the equivalence methodology to generate various online use cases; The online test cases and the test cases are input into the system to be evaluated for testing to obtain test case comparison results, wherein the test case comparison results include the execution success rate, system resource consumption and response time of the online test cases and the test cases; The use case feature analysis results of the system to be evaluated are determined based on the use case comparison results. The step of determining the traffic characteristic analysis results of the system to be evaluated based on the traffic data includes: The traffic data is anonymized to generate anonymized traffic data; The anonymized traffic data is replayed on the system to be evaluated to determine the traffic trend indicators of the system to be evaluated. The traffic characteristic analysis results are determined based on the traffic trend indicators, which include traffic trend, average service time, and database connection pool.

2. The method according to claim 1, characterized in that, The collected traffic data includes: The business system before the upgrade is regarded as the old business system, and the business system after the upgrade is regarded as the system to be evaluated. Obtain a collection instruction, wherein the collection instruction includes the collection time and the service type; Based on the collection command, online traffic is collected from the old business system to obtain traffic data.

3. The method according to claim 1, characterized in that, The step of determining the system upgrade evaluation result based on the traffic characteristic analysis result and the use case characteristic analysis result includes: Based on the traffic characteristic analysis results and the use case characteristic analysis results, determine the current program coverage of the system to be evaluated, and obtain the historical program coverage. The program coverage analysis results are determined based on the current program coverage and the historical program coverage. The traffic feature analysis results, the use case feature analysis results, and the program coverage analysis results are visualized to generate system upgrade evaluation results.

4. The method according to claim 3, characterized in that, After obtaining the historical program coverage, the method further includes: When the current program coverage is less than the historical program coverage, a customized use case is obtained based on the current coverage. The system to be evaluated is tested based on the customized use cases.

5. A business system upgrade evaluation device, characterized in that, include: The traffic characteristic analysis result determination module is used to collect traffic data and determine the traffic characteristic analysis result of the system to be evaluated based on the traffic data. The use case feature analysis result determination module is used to acquire test cases and determine the use case feature analysis result of the system to be evaluated based on the test cases and the traffic data. The system upgrade evaluation result determination module is used to determine the system upgrade evaluation result based on the traffic characteristic analysis result and the use case characteristic analysis result. The use case feature analysis result determination module is specifically used for: classifying the traffic data according to the size range, business type generating the traffic data, and traffic flow direction using an equivalence methodology to generate online use cases; inputting the online use cases and the test cases into the system to be evaluated for testing to obtain use case comparison results, wherein the use case comparison results include the execution success rate, system resource consumption, and response time of the online use cases and test cases; and determining the use case feature analysis results of the system to be evaluated based on the use case comparison results. Specifically, the traffic feature analysis result determination module is used to: perform data anonymization on the traffic data to generate anonymized traffic data; replay the anonymized traffic data in the system to be evaluated to determine the traffic trend indicators of the system to be evaluated; and determine the traffic feature analysis results based on the traffic trend indicators, wherein the traffic trend indicators include traffic trend, average service time, and database connection pool.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.

7. A computer storage medium, characterized in that, The computer storage medium stores computer instructions that are used to cause the processor to execute the method of any one of claims 1-4.

8. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-4.

Images