Test method and device for business system change

Abstract

The embodiment of the application provides a kind of service system change test method and device, the method comprises: listening message queue, to obtain the change message corresponding to the change event of service system from message queue, message queue is used to write the change message corresponding to change event when there is change event in service system.The change event corresponding to target test case is obtained in multiple preset test cases according to change message.According to target test case, test operation is executed to changed service system, to obtain the test result corresponding to change event.The technical scheme of the application can effectively improve the test efficiency of changed service system.

Description

Technical Field

[0001] This application relates to computer technology, and more particularly to a testing method and apparatus for changes in a business system. Background Technology

[0002] Making configuration changes in business systems is a common experimental method. Testing the changed business systems can effectively verify the effectiveness of the configuration changes.

[0003] The common approach is to manually trigger and execute the test process after the configuration changes to the business system have been released, in order to verify the effectiveness of the configuration changes and determine whether the business system has achieved the expected results.

[0004] However, manually triggering and executing the test process incurs a certain time cost and has relatively low operational efficiency. Summary of the Invention

[0005] This application provides a testing method and apparatus for business system changes to overcome the problem of low testing efficiency for business system changes.

[0006] In a first aspect, embodiments of this application provide a testing method for changes to a business system, including:

[0007] Listen to the message queue to retrieve the change message corresponding to the change event of the business system from the message queue. The message queue is used to write the change message corresponding to the change event when there is a change event in the business system.

[0008] Based on the change message, obtain the target test case corresponding to the change event from multiple preset test cases;

[0009] The modified business system is tested according to the target test cases to obtain the test results corresponding to the change events.

[0010] Secondly, embodiments of this application provide a testing apparatus for changes to a business system, comprising:

[0011] The processing module is used to listen to the message queue to obtain the change message corresponding to the change event of the business system from the message queue. The message queue is used to write the change message corresponding to the change event when there is a change event in the business system.

[0012] The acquisition module is used to acquire the target test case corresponding to the change event from multiple preset test cases based on the change message;

[0013] The testing module is used to perform test operations on the modified business system according to the target test cases in order to obtain the test results corresponding to the change event.

[0014] Thirdly, embodiments of this application provide an electronic device, including:

[0015] Memory, used to store programs;

[0016] A processor for executing the program stored in the memory, wherein, when the program is executed, the processor is configured to perform the method described in the first aspect above and any of the various possible designs of the first aspect.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the methods described in the first aspect above and any of the various possible designs of the first aspect.

[0018] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect above and any of the various possible designs of the first aspect.

[0019] This application provides a testing method and apparatus for business system changes. The method includes: monitoring a message queue to retrieve change messages corresponding to change events in the business system, wherein the message queue is used to write change messages corresponding to change events when a change event occurs in the business system. Based on the change messages, target test cases corresponding to the change events are retrieved from multiple preset test cases. Test operations are performed on the changed business system according to the target test cases to obtain test results corresponding to the change events. When a change event occurs in the business system, the change messages of the change events are written to the message queue. Then, by monitoring the message queue to automatically retrieve change messages from the message queue, and completing automated testing operations on the changed business system based on the change messages, the test results are sent to the target device corresponding to the change event. This enables a fully automated processing flow after a change in the business system, thereby effectively improving the testing efficiency for the changed business system. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1A schematic diagram illustrating a scenario of configuration changes to a business system provided in an embodiment of this application;

[0022] Figure 2 A flowchart of a testing method for business system changes provided in embodiments of this application;

[0023] Figure 3 The flowchart of the testing method for business system changes provided in the embodiments of this application Figure 2 ;

[0024] Figure 4 A schematic diagram illustrating the implementation of the preset mapping relationship provided in the embodiments of this application;

[0025] Figure 5 This is a schematic diagram illustrating the implementation of filtering change messages provided in an embodiment of this application.

[0026] Figure 6 A flowchart illustrating the testing method for business system changes provided in this application;

[0027] Figure 7 A schematic diagram of the structure of the testing device for business system changes provided in the embodiments of this application;

[0028] Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] To better understand the technical solution of this application, the relevant technologies involved in this application will be further described in detail below.

[0031] Service-oriented products often have configuration change capabilities. These capabilities can be understood as changing business configurations or data within the product's business system to achieve the desired effect. Since configuration changes only require simple modifications to the corresponding configurations, the switching between old and new functions can be achieved without a release.

[0032] To ensure the orderly management of configuration changes to business systems, experiments can be used. For example, an experiment can be launched targeting the business system with its configuration changes, and then the experiment can be tested to determine the effectiveness of the changes. The effectiveness of configuration changes has two aspects: firstly, the business system can operate normally and stably after the configuration changes; secondly, the business system can achieve the expected effects or functions after the configuration changes.

[0033] For example, we can understand this using A / B testing. When developers need to make changes to a business system, they can deploy A / B experiments to verify the effects of different changes. A / B testing allows for the accurate selection of solutions that maximize the value of local metrics. Ensuring stable and effective A / B testing within the business system is a crucial aspect of quality assurance. Reducing the verification cost and improving efficiency of experimental changes in diverse testing scenarios is key to quality assurance.

[0034] Based on the above introduction, it can be determined that an experiment released for a business system is actually aimed at some changes to the business system. The content of the experiment may be to add some functions to the business system, or to modify some deployments in the business system. The specific experimental content can be selected and set according to actual needs, and this embodiment does not limit it.

[0035] Based on the above introduction, for example, we can combine... Figure 1 Understanding the configuration changes to the business system. Figure 1 This is a schematic diagram illustrating a scenario of configuration changes for a business system provided in an embodiment of this application.

[0036] In this embodiment, the business system is divided into at least one business line, and each business line includes at least one application, where each application is understood as a service. For example, in a map scenario, there is a service that plans a route from location 1 to location 2. This route planning service can be understood as an application. One application in the business system can be understood as existing as a software unit to provide corresponding service support for related products. Furthermore, each application is configured with at least one experiment, which involves changes to some configurations of that application.

[0037] For example, Figure 1 As shown, a business system may include, for example, business line A, business line B, and business line C. Each business line may include application A and application B, and business line B may include application C. (See also...) Figure 1In application B, for example, experiments A, B, and C can be configured.

[0038] Taking Experiment A as an example, Experiment A might involve changing the configuration of application B within business line A. To determine the effect of the configuration change, Experiment A needs to be tested, and the test results will then determine whether the configuration change is effective.

[0039] In actual implementation, the specific business line division, specific application settings, and specific experiment configurations can all be selected and set according to actual needs, and this embodiment does not impose any restrictions on them.

[0040] Based on the above introduction, it is clear that testing the experiment after its release to the business system is crucial. The test results determine the effectiveness of the configuration changes implemented in the experiment. Currently, the verification of the effectiveness of configuration changes typically involves manually observing the completion of the experiment release and then manually triggering and executing the testing process for the changed business system. However, this manual triggering and execution method incurs time costs and is relatively inefficient.

[0041] It is understandable that when a new experiment is added to a business system, the experiment needs to be tested, and when an existing experiment is adjusted, the adjusted experiment also needs to be further tested. If each adjusted experiment needs to be manually observed and released and the test is manually triggered, it will undoubtedly lead to a significant reduction in the testing efficiency of the experiment.

[0042] To address the problems in the existing technology, this application proposes the following technical concept: an automated process can be designed to automatically trigger the execution of subsequent testing processes after the experimental release is completed, and after obtaining the test results, the test results can be directly reported to the relevant personnel, thereby enabling fully automated testing of configuration changes to business systems without manual intervention.

[0043] Based on the above description, the testing method for business system changes provided in this application will be described in detail below with reference to specific embodiments. It should be noted that the execution subject of each embodiment in this application can be a server, processor, computer, chip, or other device with data processing capabilities. This embodiment does not limit this; any device with data processing capabilities can serve as the execution subject of each embodiment in this application.

[0044] The following is a combination of... Figure 2 This application introduces the testing methods for business system changes. Figure 2A flowchart of a testing method for business system changes provided in an embodiment of this application.

[0045] like Figure 2 As shown, the method includes:

[0046] S201. Listen to the message queue to retrieve the change message corresponding to the change event of the business system. The message queue is used to write the change message corresponding to the change event when there is a change event in the business system.

[0047] In this embodiment, developers can, for example, make configuration changes to the business system. After a configuration change is made to the business system, an experiment can be released for the business system. This experiment can be a newly added experiment or an adjustment to an existing experiment.

[0048] It can also be determined that when the business system undergoes a configuration change, a change event can be considered to exist. Based on the above description, the change event can be a new experiment, or it can be a change to an existing experiment. This embodiment does not limit the specific implementation of the change event. Any event that changes the configuration of the business system can be considered a change event in this embodiment.

[0049] For example, following the examples above, if developers make configuration changes to the business system, resulting in the addition of an experiment D, then a change event can be considered to exist, corresponding to the addition of experiment D. Alternatively, if developers make configuration changes to the business system, thereby adjusting the original experiment A to obtain experiment A', then a change event can also be considered to exist, corresponding to the adjustment of experiment A to experiment A'.

[0050] In one possible implementation, after the business system undergoes a configuration change, a change message corresponding to the change event can be generated and written to a message queue. This process of generating and writing the change message to the message queue can be accomplished, for example, through an experimental platform.

[0051] The processing device in this embodiment can monitor the message queue. When a message is detected in the message queue, it can consume the message in the message queue to obtain the change message corresponding to the change event of the business system. The change message corresponding to the change event may include relevant event data, and the specific data content of the event data can be selected and set according to actual needs.

[0052] S202. Based on the change message, obtain the target test case corresponding to the change event from multiple preset test cases.

[0053] In this embodiment, multiple test cases can be preset. After obtaining the change message corresponding to the change event, the target test case corresponding to the change event can be obtained from multiple preset test cases based on the change message.

[0054] In one possible implementation, the change message may include event information corresponding to the change event. For example, a mapping relationship between event information and preset test cases may be pre-set. Then, matching can be performed based on the mapping relationship to obtain the target test case corresponding to the change event from multiple preset test cases.

[0055] Alternatively, the matching degree can be determined based on the event information in the change message and the test case information, thereby identifying the target test case corresponding to the change event. This embodiment does not limit the specific implementation method for determining the target test case, as long as the target test case is determined from multiple preset test cases and is the test case corresponding to the change event.

[0056] S203. Perform test operations on the modified business system according to the target test cases to obtain the test results corresponding to the change events.

[0057] In this embodiment, after a configuration change is made to the business system, the changed business system needs to be tested to determine the effectiveness of the configuration change. Therefore, after determining the target test cases corresponding to the change event, test operations can be automatically executed on the changed business system according to the target test cases, thereby obtaining the test results corresponding to the change event. The test results corresponding to the change event are actually the test results of the experiment corresponding to the change event.

[0058] If the test results indicate that the changed business system can operate stably and achieve the expected goals, then the test result corresponding to the change event can be determined as successful. Conversely, if the test results indicate that the changed business system cannot operate stably or cannot achieve the expected goals, then the test result corresponding to the change event can be determined as failed.

[0059] After determining the test results corresponding to the change event, the test results can be sent to the target device corresponding to the change event. Alternatively, the target device can actively obtain the test results. The specific test result reporting method can be selected and set according to actual needs.

[0060] The target device corresponding to the change event can be, for example, the device of the initiator of the experiment corresponding to the change event, or the device of the person in charge of the experiment corresponding to the change event. This embodiment does not limit this.

[0061] In this embodiment, sending the test results obtained above to the target device corresponding to the change event enables the automated execution of a series of test processes after detecting a configuration change in the business system, and finally automatically reports the obtained test results to the relevant personnel, thereby achieving rapid and effective testing of configuration changes in the business system.

[0062] The testing method for business system changes provided in this application includes: listening to a message queue to retrieve change messages corresponding to change events in the business system, wherein the message queue is used to write change messages corresponding to change events when a change event occurs in the business system; retrieving target test cases corresponding to the change event from multiple preset test cases based on the change messages; and performing test operations on the changed business system according to the target test cases to obtain test results corresponding to the change events. When a change event occurs in the business system, the change messages of the change events are written to the message queue, and then the message queue is listened to to automatically retrieve change messages from the message queue. Automated testing operations on the changed business system are then performed based on the change messages, thereby achieving a fully automated processing flow after a change in the business system and effectively improving the testing efficiency for the changed business system.

[0063] Based on the above introduction, it can be determined that after obtaining the change message from the message queue, it is necessary to retrieve the target test case corresponding to the change event from multiple preset test cases according to the change message. Building upon the above, the following section will combine... Figures 3 to 5 The implementation method for determining the target test cases in the testing method for business system changes provided in this application will be described in further detail. Figure 3 The flowchart of the testing method for business system changes provided in the embodiments of this application Figure 2 , Figure 4 This is a schematic diagram illustrating the implementation of the preset mapping relationship provided in the embodiments of this application. Figure 5 This is a schematic diagram illustrating the implementation of filtering change messages provided in an embodiment of this application.

[0064] like Figure 3 As shown, the method includes:

[0065] S301. Determine whether there is a change business line identifier in the business line identifier set. If yes, execute S302; otherwise, execute S307.

[0066] Based on the above description, it can be determined that a change event can be a newly added experiment or an adjustment to an existing experiment. Each experiment belongs to a specific application, and each application belongs to a specific business line. Therefore, in this embodiment, the change message corresponding to a change event may include, for example, the change business line identifier corresponding to the change event, the change application identifier corresponding to the change event, and the change experiment identifier corresponding to the change event.

[0067] Among them, the change experiment corresponding to the change event belongs to the change application, and the change application corresponding to the change event belongs to the change business line. In other words, the change business line identifier, change application identifier, and change experiment identifier corresponding to the change event follow the specific affiliation relationship of experiment, application, and business line.

[0068] For example, as mentioned above Figure 1 For example, assuming the current change event is an adjustment to experiment A, we can determine that the change experiment identifier corresponding to this change event is experiment A, the change application identifier corresponding to this change event is application B, and the change business line identifier corresponding to this change event is business line A.

[0069] After obtaining the change message corresponding to the change event, it is necessary to obtain the target test case corresponding to the change event from multiple preset test cases based on the change message.

[0070] In one possible implementation, for example, a set of business line identifiers can be pre-set, which includes all business lines in the business system. In order to ensure that the target test case corresponding to the change event can be effectively determined, it can be first determined whether the change business line identifier included in the change message exists in the set of business line identifiers.

[0071] In one possible implementation, if no change business line identifier exists in the business line identifier set, it can be determined that the business line corresponding to the current change event is not a business line included in the business system. In such a scenario, the event information itself may be incorrect, or the system may have encountered an anomaly. In short, if no change business line identifier can be matched for the change event, the change message corresponding to the current change event can be directly ignored. Furthermore, a notification message can be sent to the target device to inform it that the business line corresponding to the current change event is not a business line existing in the business system.

[0072] S302. Determine whether there are test cases corresponding to the changed experimental identifier based on the preset mapping relationship. If yes, execute S303; otherwise, execute S304.

[0073] In one possible implementation, if a changed business line identifier exists in the set of business line identifiers, then the business line corresponding to the current change event can be considered to be a business line existing in the business system, and the subsequent target test case matching operation can continue to be executed.

[0074] In this embodiment, a preset mapping relationship can be set, which indicates the correspondence between preset test cases, experiment identifiers, and application identifiers. This correspondence can be represented, for example, as application -> experiment -> test case. For example, it can be combined with... Figure 4 To understand, such as Figure 4 As shown, assuming the preset mapping relationship includes a correspondence between Application B, Experiment A, and Test Case 1, and a correspondence between Application B, Experiment B, and Test Case 2, the rest are implemented similarly and will not be elaborated here. In actual implementation, the correspondence between test cases, experiments, and applications included in the preset mapping relationship can be selected and set according to actual needs.

[0075] In this embodiment, the change message includes a change experiment identifier corresponding to the change event. Based on the determination that there is a change business line identifier in the business line identifier set, it can be further determined whether there is a test case corresponding to the change experiment identifier according to the preset mapping relationship described above.

[0076] S303. Determine the test case corresponding to the change experiment identifier in the preset mapping relationship as the target test case.

[0077] In one possible implementation, if the preset mapping relationship includes a correspondence between change experiment identifiers and corresponding test cases, then the existence of test cases corresponding to the change experiment identifiers can be determined based on the preset mapping relationship. In other words, test cases corresponding to the experiment to which the current change event belongs are pre-configured. Therefore, the test cases corresponding to the change experiment identifiers in the preset mapping relationship can be directly determined as the target test cases.

[0078] For example, if the experiment identifier corresponding to the change event in the change message is experiment A, then according to Figure 4 The preset mapping relationship shown can determine that test case 1 corresponding to experiment A is the target test case.

[0079] S304. Determine whether there are test cases corresponding to the changed application identifier based on the preset mapping relationship. If yes, execute S305; otherwise, execute S306.

[0080] In another possible implementation, if the preset mapping relationship does not include the correspondence between the change experiment identifier and the corresponding test case, then it can be determined from the preset mapping relationship that there is no test case corresponding to the change experiment identifier. In other words, the test case corresponding to the experiment to which the current change event belongs has not been configured in advance.

[0081] In this case, it can be further determined whether there are test cases corresponding to the changed application identifier based on the preset mapping relationship.

[0082] In this embodiment, the change message includes the change application identifier corresponding to the change event. Based on the premise that there is a change business line identifier in the business line identifier set and there is no test case corresponding to the change experiment identifier, it can be further determined whether there is a test case corresponding to the change application identifier according to the preset mapping relationship described above.

[0083] S305. Determine the test case corresponding to the application identifier change in the preset mapping relationship as the target test case.

[0084] In one possible implementation, if the preset mapping relationship includes a correspondence between the changed application identifier and the corresponding test case, then the existence of a test case corresponding to the changed application identifier can be determined based on the preset mapping relationship. In other words, test cases corresponding to the application to which the current change event belongs are pre-configured. Therefore, the test case corresponding to the changed application identifier in the preset mapping relationship can be directly determined as the target test case.

[0085] If there is only one test case corresponding to the change of application identifier in the preset mapping relationship, then this single test case can be directly identified as the target test case. Alternatively, if there are multiple test cases corresponding to the change of application identifier in the preset mapping relationship, one or more of these test cases can be selected as the target test case. The selection method can be, for example, random selection or selection according to priority; this embodiment does not impose any restrictions on this.

[0086] Here, we'll illustrate this with a specific example. Suppose the current change event involves adding experiment D to application B. Following the processing flow described above, we can determine that experiment D belongs to application B, and application B belongs to business line A. Therefore, we can first determine that business line A exists in the business line identifier set.

[0087] And referring to the above introduction Figure 4 It is certain that the pre-defined mapping does not include the correspondence between experiment D and its corresponding test cases. Therefore, we can take a step back and determine whether there are test cases corresponding to application B based on the pre-defined mapping, referring to... Figure 5 It can be determined that the test cases corresponding to the current application B include test case 1, test case 2, and test case 3. Therefore, for example, one or more of these three test cases can be selected and designated as the target test case.

[0088] After identifying the target test cases, for example, a notification message can be sent to the target device. This notification message indicates that the experiment corresponding to the current change experiment identifier has been changed and that its automated testing has started, so that relevant personnel can keep track of the test progress in a timely manner.

[0089] It's important to note that in the current scenario, Experiment D is a newly added experiment, and there's no pre-defined mapping between Experiment D and its corresponding test cases. This indicates that no test cases have been pre-configured for Experiment D. In this case, to ensure successful automated testing of Experiment D, we can opt for test cases corresponding to Application B, which contains Experiment D. Since test data for an application is generally similar, these test cases can effectively test Experiment D. However, compared to test cases specifically configured for Experiment D, the compatibility of the currently determined test cases with Experiment D is slightly inferior.

[0090] It's understandable that configuring corresponding test cases for each experiment would yield the best testing results, as this ensures that specific test cases are tailored to each experiment. However, in special cases where no corresponding test cases are configured for certain experiments, using the test cases corresponding to the application to which the experiment belongs can still guarantee the smooth progress of automated testing.

[0091] S306. Send a notification message to the target device. The notification message is used to indicate the target test case for which no change event exists.

[0092] In another possible implementation, if the second mapping relationship determines that there is no test case corresponding to the changed application identifier, then it can be considered that the test case corresponding to the change event cannot be automatically matched. In this case, a notification message can be sent to the target device, indicating that there is no target test case corresponding to the change event, after which developers can perform further processing.

[0093] S307, Ignore change messages.

[0094] The testing method for business system changes provided in this application first determines whether the business line corresponding to the change event is a business line within the business system. If so, subsequent matching processing is performed, effectively avoiding the need for further processing of abnormal situations and saving resources consumed in handling abnormal data. Furthermore, during the subsequent matching process, when a test case corresponding to the experiment to which the change event belongs is identified, the test case corresponding to the experiment is prioritized as the target test case, thus accurately matching the test cases corresponding to the changed business line and the change experiment. If no test case corresponding to the change experiment is identified, the test case corresponding to the application to which the change event belongs is matched as a fallback, and then identified as the target test case for subsequent automated testing, thereby effectively improving the success rate of matching target test cases.

[0095] Building upon the above, after retrieving change messages from the message queue, these messages can be filtered. This filtering process can remove messages that don't require automated testing, thereby improving the efficiency and effectiveness of the automated testing process. The following section will combine... Figure 5 To provide a more detailed introduction, Figure 5 This is a message filtering diagram illustrating the testing method for business system changes provided in this application embodiment.

[0096] like Figure 5 As shown, after retrieving a change message from the message queue, change messages whose message type is not a preset type can be filtered out.

[0097] In one possible implementation, the change message may include a message type, or the change message may carry its own message type, in which case the message type of the change message can be obtained.

[0098] When the message type of a change message is not a preset type, it can be assumed that the change message is not a message that needs to trigger automated testing, and the change message can be discarded directly.

[0099] Alternatively, if the message type of the change message is a preset type, then the change message can be considered a message that needs to trigger automated testing, and subsequent processing can be carried out based on the change message.

[0100] In actual implementation, message types may include, for example, experimental change type, pre-change type, process initiation type, and status reset type. If only the experimental change type needs to trigger subsequent automated tests, then the preset type can be set to experimental change type. In actual implementation, the specific message type classification and preset types can be selected and set according to actual needs. Any message type that needs to trigger automated tests can be used as the preset type in this embodiment.

[0101] And in one possible implementation, such as Figure 5 As shown, in this embodiment, for example, test activation information can also be configured, which is used to indicate whether the automated test corresponding to the change event is enabled. The test activation information is, for example, a switch-like information. When the test activation information indicates that the automated test corresponding to the change event is not enabled, it can be assumed that the current change event does not need to trigger automated testing, and the change message can be discarded.

[0102] Alternatively, when the test start information indicates that the automated test corresponding to the change event has been started, it can be assumed that the automated test corresponding to the change event needs to be triggered, and subsequent processing can be carried out according to the change message.

[0103] The test enable information can be configured globally, for example, by setting it to "disabled" globally, meaning that no automated tests will be triggered by any change events. Alternatively, the test enable information can be configured for an application, such as setting it to "disabled" for application A, which ensures that no automated tests will be triggered by any change events corresponding to experiments under application A. The test enable information can also be configured for business lines or experiments, etc. This embodiment does not limit the specific implementation method. In short, based on the test enable information and the corresponding information of the change event, it can be determined whether the test enable information specifically indicates whether the automated test corresponding to the change event is enabled or disabled.

[0104] When the test enable information indicates that the automated test corresponding to the change event is disabled, after the change event occurs, for example, an instruction information can be sent to the target device corresponding to the change event to inform that a change event has occurred and the corresponding automated test process has not been triggered.

[0105] In another possible implementation, to increase the flexibility of experiment configuration, a blacklist mechanism could be provided. By adding certain experiments to the blacklist, change events corresponding to experiments on the blacklist would not trigger the automated testing process. Therefore, upon receiving a change message, if it is determined that the change message belongs to an experiment on the blacklist, the change message can be discarded.

[0106] Based on the above introduction and Figure 5 It is certain that in this embodiment, filtering of acquired change messages can be achieved by determining the message type, determining whether the automated test corresponding to the change event is enabled through test activation information, and implementing the experiment blacklist. These three filtering methods are not sequential and can be set according to actual needs. By filtering messages in the message queue, it can be ensured that automated tests are triggered only for change messages that require them, thus effectively avoiding server overhead caused by frequent calls to corresponding interfaces and also avoiding database queries. Furthermore, the implementation method of configuring test activation information described above can prevent interface call errors or erroneous data caused by test case execution when the service experiences circuit breaking or degradation. The test activation information allows for flexible and effective configuration of whether automated tests are triggered.

[0107] After determining the test results corresponding to the change event, abnormal test results usually need to be reported to the relevant responsible personnel. However, the cause of some automated tests may not be due to the experiment change, but rather to problems with the test data. If a notification is sent for every test failure, it may lead to frequent interruptions to the responsible personnel.

[0108] Therefore, in this embodiment, the experiment corresponding to the change event can be called the change experiment. Then, the total number of test cases executed for the change experiment is determined, as well as the number of test cases that failed among the test cases executed for the change experiment. Then, for example, the test failure ratio corresponding to the change experiment can be determined based on the ratio of the latter to the former.

[0109] For example, if Experiment A was tested with a total of 10 test cases, and 2 of those test cases failed, then the failure rate of Experiment A can be determined to be 20%.

[0110] When the failure rate of the test corresponding to the change experiment is greater than or equal to the preset threshold, an alarm message is sent to the target device. The alarm message is used to indicate that there is an anomaly in the test corresponding to the change experiment.

[0111] In one possible implementation, when sending alarm information to the target device, the alarm can be triggered via a preset method, such as a telephone call. In other words, this alarm information is more informative than the notification information for reporting test results described above, thus ensuring that relevant personnel can promptly detect and intervene in any abnormal issues.

[0112] Understandably, this method allows for notification to the responsible personnel only when the failure rate of the test corresponding to the change experiment reaches a certain percentage, thus avoiding the disturbance caused by frequent anomaly notifications.

[0113] Current related technologies, besides requiring manual triggering of the test process, leading to low testing efficiency, also suffer from the inability to automatically record experimental data and test results, thus hindering the summarization of multi-dimensional quality data and impeding the summarization and optimization of business quality development. Building on this, the business system change testing method provided in this application can further summarize and automate the analysis of test results, which will be further detailed below.

[0114] In this embodiment, after generating the test result, for example, the generation time corresponding to the test result can be obtained. Then, based on the generation time of the test result, the test result itself, and the change message corresponding to the change event, a record message corresponding to the change event can be generated. This record message can then be stored in a preset storage space.

[0115] The recorded message can be in the following form: {Change message corresponding to the change event, test result, and the time when the test result was generated}. A recorded message can be generated for each change event in this way, and the specific content of the recorded message can be selected and set according to actual needs; this embodiment does not impose any restrictions on this.

[0116] The preset storage space contains the record messages corresponding to each of the multiple change events. Then, the test statistics information within the target time period can be determined based on the record messages stored in the preset storage space.

[0117] The target time period can be set according to actual needs. When filtering based on the target time period, you can filter the generated record messages that are generated within the target time period, and then perform further filtering on this basis.

[0118] Test statistics can include, for example, test frequency, which refers to the number of tests triggered within a target time period. One possible implementation is to calculate the test frequency globally (i.e., across all business lines) within the target time period, or specifically for a particular business line, or even for a specific test team. The specific allocation of test frequency can be selected and configured according to actual needs.

[0119] Test statistics can also include, for example, the percentage of failed tests corresponding to the target business line. The percentage of failed tests can be the ratio of the total number of test cases that failed within the target time period to the number of test cases that failed within the target business line within the target time period.

[0120] Assuming that the total number of test cases that failed within the current target time period is 40, of which 14 test cases that failed correspond to the target business line "Business Line A", then the failure rate of the test cases corresponding to the target business line "Business Line A" is 35%.

[0121] The test statistics may also include the percentage of failed tests for the target application, the percentage of failed tests for the target experiment, etc. The meaning of the percentage of failed tests is similar and will not be elaborated here.

[0122] The test statistics can also include a list of experiments that failed. For example, it can list the experiments that failed within a target time period, as well as the total number of test cases for these experiments and the number of failed test cases.

[0123] The test statistics can also include the frequency of configuration changes to the experiment within the target time period. For example, it can list the number of configuration changes to the target experiment in different time periods. For instance, the target experiment's configuration was changed 12 times in October, 21 times in November, and so on.

[0124] The test statistics can also include the ranking of the configuration change frequency of multiple experiments within the target time period. For example, the experiment with the highest configuration change frequency within the target time period is experiment A, the experiment with the second highest configuration change frequency is experiment B, and so on.

[0125] In actual implementation, the specific implementation method of test statistics can be selected and set according to actual needs. Any statistical data obtained from the analysis of recorded messages stored in the preset storage space, and any statistical data required by the relevant personnel, can be used as test statistics in this embodiment. By collecting experimental test results and basic data on experimental changes, and generating recorded information, multi-dimensional quality data can be customized based on the recorded information. This allows for timely analysis and discovery of key issues, providing reliable data for subsequent adjustments to business quality. This maximizes the recording and statistical analysis of test data, thereby improving data utilization.

[0126] Based on the above introduction, the following will combine... Figure 6 To introduce, Figure 6 A flowchart illustrating the testing method for business system changes provided in this application.

[0127] like Figure 6 As shown, when there is a change event in the experiment for the business system, a message queue can be sent. Then, for example, the configuration platform can listen to the message queue and obtain the change message corresponding to the change event from the message queue.

[0128] After obtaining the change message, it can first be filtered. The specific implementation of the filtering process can be found in the above embodiment. For the remaining change messages after filtering, the relevant configuration data in the change message can be obtained, such as the change business line identifier, change application identifier, and change experiment identifier corresponding to the change event described above.

[0129] Then, the target test cases can be determined based on the relevant configuration data in the change message, and the automated test process can be executed. After the automated test is triggered, for example, a message notification can be sent to the corresponding target device, and the test results of the automated test can be obtained through the callback interface.

[0130] Then, based on the test results, corresponding notification information can be sent to the target device. This notification information may include, for example, a report of the test results as described in the above embodiments, or an alarm information of test failure as described in the above embodiments.

[0131] Furthermore, based on the test results, log information corresponding to change events can be generated, and then the log information can be stored. Based on the stored log information, custom test statistics can be further determined to analyze the risks of possible configuration changes.

[0132] In summary, the testing method for business system changes provided in this application can, on the one hand, automatically trigger corresponding test cases immediately after configuration changes are completed by setting up a series of automated processes such as message queues, eliminating the manual triggering costs associated with manual intervention. Furthermore, it can promptly send alarm information to the target device when certain conditions are met, reducing losses caused by untimely problem detection and effectively improving the testing efficiency of experimental changes. Simultaneously, by storing the record information corresponding to the change events, it facilitates subsequent customized data measurement and statistics, providing effective data references for adjustments to the business system.

[0133] Figure 7 A schematic diagram of the structure of a test apparatus for business system changes provided in an embodiment of this application. (See attached diagram.) Figure 7 As shown, the device 70 includes: a processing module 701, an acquisition module 702, a testing module, and a sending module 704.

[0134] The processing module 701 is used to listen to the message queue to obtain the change message corresponding to the change event of the business system from the message queue. The message queue is used to write the change message corresponding to the change event when there is a change event in the business system.

[0135] The acquisition module 702 is used to acquire the target test case corresponding to the change event from multiple preset test cases based on the change message;

[0136] The test module 703 is used to perform test operations on the modified business system according to the target test cases in order to obtain the test results corresponding to the change event.

[0137] In one possible design, the change message corresponding to the change event includes: the change business line identifier corresponding to the change event, the change application identifier corresponding to the change event, and the change experiment identifier corresponding to the change event;

[0138] The business system includes at least one business line, each business line includes at least one application, each application is configured with at least one experiment, and the change event is a configuration change for an existing experiment or a new experiment.

[0139] In one possible design, the acquisition module 702 is specifically used for:

[0140] If it is determined that the changed business line identifier exists in the business line identifier set, then it is determined whether there is a test case corresponding to the changed experiment identifier according to the preset mapping relationship. The preset mapping relationship is used to indicate the correspondence between the preset test case, experiment identifier and application identifier.

[0141] If it exists, then the test case corresponding to the changed experiment identifier in the preset mapping relationship is determined as the target test case; or,

[0142] If it does not exist, the target test case corresponding to the change event is obtained from multiple preset test cases based on the change application identifier.

[0143] In one possible design, the acquisition module 702 is specifically used for:

[0144] Determine whether there is a test case corresponding to the changed application identifier based on the preset mapping relationship;

[0145] If it exists, the test case corresponding to the changed application identifier in the preset mapping relationship will be determined as the target test case.

[0146] In one possible design, the transmitting module 704 is used for:

[0147] A notification message is sent to the target device, the notification message being used to instruct the automated testing of the experiment corresponding to the changed experiment identifier to begin execution.

[0148] In one possible design, the sending module 704 is further configured to:

[0149] If there is no test case corresponding to the changed application identifier in the preset mapping relationship, a notification message is sent to the target device. The notification message is used to indicate that there is no target test case corresponding to the changed event.

[0150] In one possible design, the processing module 701 is further configured to:

[0151] After obtaining the change message corresponding to the change event of the business system, obtain the message type of the change message;

[0152] If the message type of the change message is not a preset type, the change message is discarded.

[0153] In one possible design, the processing module 701 is further configured to:

[0154] After obtaining the change message corresponding to the change event of the business system, test start information is obtained, which is used to indicate whether the automated test corresponding to the change event is started.

[0155] If the test activation information indicates that the automated test corresponding to the change event is not enabled, then the change message is discarded.

[0156] In one possible design, the processing module 701 is further configured to:

[0157] After performing test operations on the modified business system according to the target test cases to obtain the test results corresponding to the change event, the test failure ratio corresponding to the change experiment is determined based on the total number of test cases executed for the change experiment and the number of test cases with failed test results among the test cases executed for the change experiment. The change experiment is the experiment corresponding to the change event.

[0158] If the test failure rate is greater than or equal to a preset threshold, an alarm message is sent to the target device. The alarm message is used to indicate the test anomaly corresponding to the change experiment.

[0159] In one possible design, the processing module 701 is further configured to:

[0160] After performing test operations on the modified business system according to the target test cases to obtain the test results corresponding to the change event, the generation time corresponding to the test results is obtained;

[0161] Based on the generation time corresponding to the test result, the test result, and the change message of the change event, a record message corresponding to the change event is generated and stored in a preset storage space.

[0162] In one possible design, the processing module 701 is further configured to:

[0163] After storing the recorded messages, test statistics for the target time period are determined based on the recorded messages stored in the preset storage space.

[0164] The test statistics include at least one of the following: test frequency, percentage of failed tests corresponding to the target business line, and a list of experiments with failed test results.

[0165] The apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0166] Figure 8 A schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application, such as... Figure 8 As shown, the electronic device 80 of this embodiment includes: a processor 801 and a memory 802; wherein

[0167] Memory 802 is used to store instructions executed by the computer;

[0168] The processor 801 is used to execute computer execution instructions stored in the memory to implement the various steps of the business system change testing method in the above embodiments. For details, please refer to the relevant descriptions in the foregoing method embodiments.

[0169] Alternatively, the memory 802 can be either standalone or integrated with the processor 801.

[0170] When the memory 802 is set up independently, the electronic device also includes a bus 803 for connecting the memory 802 and the processor 801.

[0171] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement a test method for business system changes performed by the aforementioned electronic device.

[0172] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0173] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application.

[0174] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0175] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0176] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0177] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0178] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A testing method for changes to a business system, characterized in that, include: The system listens to a message queue to retrieve change messages corresponding to change events in the business system. The message queue is used to write change messages corresponding to change events when there are change events in the business system. The business system includes at least one business line, each business line includes at least one application, and each application is configured with at least one experiment. The change event is a configuration change for an existing experiment or a new experiment. The change message includes: a change business line identifier corresponding to the change event, a change application identifier corresponding to the change event, and a change experiment identifier corresponding to the change event. If it is determined that the changed business line identifier exists in the business line identifier set, then it is determined whether there is a test case corresponding to the changed experiment identifier according to the preset mapping relationship. The preset mapping relationship is used to indicate the correspondence between the preset test case, experiment identifier and application identifier. If a test case corresponding to the changed experiment identifier exists, then the test case corresponding to the changed experiment identifier in the preset mapping relationship is determined as the target test case; If there is no test case corresponding to the changed experiment identifier, then determine whether there is a test case corresponding to the changed application identifier according to the preset mapping relationship; If a test case corresponding to the changed application identifier exists, then the test case corresponding to the changed application identifier in the preset mapping relationship is determined as the target test case; The modified business system is tested according to the target test cases to obtain the test results corresponding to the change events.

2. The method according to claim 1, characterized in that, After performing test operations on the modified business system according to the target test cases to obtain the test results corresponding to the change event, the method further includes: Based on the total number of test cases executed for the change experiment and the number of test cases that failed in the test cases executed for the change experiment, the test failure ratio corresponding to the change experiment is determined, where the change experiment is the experiment corresponding to the change event. If the test failure rate is greater than or equal to a preset threshold, an alarm message is sent to the target device. The alarm message is used to indicate the test anomaly corresponding to the change experiment.

3. The method according to any one of claims 1-2, characterized in that, After performing test operations on the modified business system according to the target test cases to obtain the test results corresponding to the change event, the method further includes: Obtain the generation time corresponding to the test result; Based on the generation time corresponding to the test result, the test result, and the change message of the change event, a record message corresponding to the change event is generated and stored in a preset storage space.

4. The method according to claim 3, characterized in that, After storing the recorded message, the method further includes: Based on the recorded messages stored in the preset storage space, determine the test statistics for the target time period; The test statistics include at least one of the following: test frequency, percentage of failed tests corresponding to the target business line, and a list of experiments with failed test results.

5. A testing apparatus for changes to a business system, characterized in that, include: A processing module is used to listen to a message queue to obtain change messages corresponding to change events of the business system from the message queue. The message queue is used to write change messages corresponding to change events when there are change events in the business system. The business system includes at least one business line, each business line includes at least one application, each application is configured with at least one experiment, the change event is a configuration change for an existing experiment, or the change event is a new experiment, and the change message includes: the change business line identifier corresponding to the change event, the change application identifier corresponding to the change event, and the change experiment identifier corresponding to the change event. The acquisition module is used to acquire the target test case corresponding to the change event from multiple preset test cases based on the change message; The testing module is used to perform test operations on the modified business system according to the target test cases, so as to obtain the test results corresponding to the change event; The acquisition module is specifically used for: If it is determined that the changed business line identifier exists in the business line identifier set, then it is determined whether there is a test case corresponding to the changed experiment identifier according to the preset mapping relationship. The preset mapping relationship is used to indicate the correspondence between the preset test case, experiment identifier and application identifier. If a test case corresponding to the changed experiment identifier exists, then the test case corresponding to the changed experiment identifier in the preset mapping relationship is determined as the target test case; If there is no test case corresponding to the changed experiment identifier, then determine whether there is a test case corresponding to the changed application identifier according to the preset mapping relationship; If a test case corresponding to the changed application identifier exists, then the test case corresponding to the changed application identifier in the preset mapping relationship is determined as the target test case.

6. An electronic device, characterized in that, include: Memory, used to store programs; A processor for executing the program stored in the memory, wherein when the program is executed, the processor is configured to perform the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 4.

Images