A backtesting verification method and apparatus

Abstract

This application provides a backtesting verification method and apparatus. The method includes: acquiring a first operation result generated by first code running online within a preset time, and recorded user behavior data; acquiring a second code modified from the first code, running the second code based on the user behavior data, and generating a second operation result; comparing the first operation result and the second operation result to verify whether the modification of the first code is correct. This method can ensure the accuracy of backtesting verification.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a backtesting verification method and apparatus. Background Technology

[0002] Currently, after modifying the code running online, a large number of test cases are used to test it in order to verify whether the modifications to the code running online meet expectations.

[0003] In the process of developing this application, the inventors discovered that when using test cases for testing, the verification scenarios are too small, the data is not realistic enough, and the amount of data is insufficient, which in turn cannot guarantee the accuracy of backtesting verification. Summary of the Invention

[0004] In view of this, this application provides a backtesting verification method and apparatus that can ensure the accuracy of backtesting verification.

[0005] To solve the above-mentioned technical problems, the technical solution of this application is implemented as follows:

[0006] In one embodiment, a backtesting verification method is provided, the method comprising:

[0007] Obtain the first operation result generated by the first code running online within a preset time, as well as the recorded user behavior data;

[0008] Obtain the second code after modifying the first code, run the second code based on the user behavior data, and generate a second operation result;

[0009] Compare the results of the first operation and the second operation to verify whether the modification to the first code is correct.

[0010] In another embodiment, a backtesting verification device is provided, the device comprising: a first acquisition unit, a second acquisition unit, a running unit, and an analysis unit;

[0011] The first acquisition unit is used to acquire the first operation result generated by the first code running online within a preset time, as well as the recorded user behavior data;

[0012] The second acquisition unit is used to acquire the second code after modifying the first code;

[0013] The running unit is used to run the second code obtained by the second acquisition unit based on the user behavior data obtained by the first acquisition unit, and generate a second operation result;

[0014] The analysis unit is used to compare the first operation result obtained by the first acquisition unit with the second operation result generated by the running unit to verify whether the modification of the first code is correct.

[0015] In another embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the backtesting verification method.

[0016] In another embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the backtesting verification method.

[0017] As can be seen from the above technical solution, in the above embodiments, user behavior data from the online execution of the first code within a preset time period is used as test data to test the second code in a simulated real user environment. The second code is a modified version of the first code. This solution ensures the accuracy of backtesting verification by testing the offline code with real user behavior data. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the backtesting verification process in Embodiment 1 of this application;

[0020] Figure 2 This is a schematic diagram of the backtesting verification process in Embodiment 2 of this application;

[0021] Figure 3 This is a schematic diagram of the backtesting verification process in Embodiment 3 of this application;

[0022] Figure 4 This is a schematic diagram illustrating the order volume trend in the embodiments of this application;

[0023] Figure 5 This is a schematic diagram of the backtesting verification device in the embodiments of this application;

[0024] Figure 6 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] The technical solution of the present invention will be described in detail below with reference to specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0028] This application provides a backtesting verification method, which uses user behavior data from the online execution of a first code within a preset time as test data to test a second code in a simulated real user environment. The second code is a modified version of the first code. This method of testing the offline code with real user behavior data ensures the accuracy of the backtesting verification.

[0029] The backtesting verification process in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0030] Example 1

[0031] See Figure 1 , Figure 1 This is a schematic diagram of the backtesting verification process in Embodiment 1 of this application. The specific steps are as follows:

[0032] Step 101: Obtain the first operation result generated by the first code running online within a preset time, as well as the recorded user behavior data.

[0033] When the code runs online, real users will perform user operations, resulting in corresponding operation results.

[0034] If the online operation time is relatively long, a large number of operation results and user behavior data will be generated; in this embodiment of the application, the operation results and user behavior data corresponding to the preset time period are obtained.

[0035] In the existing implementation, if the code has the function of recording user behavior data when running online, it can be recorded when the code runs. If not, this function needs to be configured to record user behavior data.

[0036] Step 102: Obtain the second code after modifying the first code, run the second code based on the user behavior data, and generate the second operation result.

[0037] Modifications to the first code can include adding new features or optimizing existing features.

[0038] Typically, after modifying code that runs online offline, backtesting is required. If the verification passes, the code can be deployed online; otherwise, the code needs to be modified again.

[0039] Therefore, the method of backtesting is crucial. If the backtesting method is inappropriate, it may not be able to effectively verify whether the modifications to the initial code meet expectations.

[0040] If backtesting is performed using test cases, but these test cases are not real data, the scenarios that can be verified are small, and the amount of data is insufficient, problems in the code may not be detected. Such backtesting will pass, and deploying such code may cause major problems.

[0041] This application does not use test cases for backtesting verification. Instead, it uses real user behavior data from the online runtime for backtesting verification, just like running the second code online. If there are problems with the second code, they can be discovered early.

[0042] Step 103: Compare the results of the first operation and the second operation to verify whether the modification of the first code is correct.

[0043] By analyzing and comparing the results of the first and second operations, such as in a shopping scenario, comparing order volume trends and purchase limits, it can be determined whether the data corresponding to the second operation result is abnormal, and then it can be determined whether the modification of the first code is correct, that is, whether it meets expectations.

[0044] In practice, the correctness of the modification to the first code is determined based on the purpose of the modification:

[0045] If the purpose of modifying the first code is to add a new function, it is necessary to determine whether the new function will affect the original function to determine whether the modification of the first code is correct.

[0046] If the purpose of modifying the first code is to optimize the original function, it is necessary to determine whether the optimized function affects other functions and whether the optimized function can be used normally, in order to determine whether the modification of the first code has achieved the expected optimization function, that is, whether the modification of the first code is correct.

[0047] In this embodiment, user behavior data from the online execution of the first code within a preset time period is used as test data to test the second code, which is a modified version of the first code, in a simulated real user environment. This approach ensures the accuracy of backtesting by testing the offline code with real user behavior data.

[0048] Example 2

[0049] See Figure 2 , Figure 2 This is a schematic diagram of the backtesting verification process in Embodiment 2 of this application. The specific steps are as follows:

[0050] Step 201: Obtain the first operation result generated by the first code running online within a preset time, as well as the recorded user behavior data.

[0051] When the code runs online, real users will perform user operations, resulting in corresponding operation results.

[0052] If the online operation time is relatively long, a large number of operation results and user behavior data will be generated; in this embodiment of the application, the operation results and user behavior data corresponding to the preset time period are obtained.

[0053] In the existing implementation, if the code has the function of recording user behavior data when running online, it can be recorded when the code runs. If not, this function needs to be configured to record user behavior data.

[0054] User behavior data includes all data information when a user accesses the internet online, such as timestamps, activity IDs, target objects, and operation types.

[0055] The timestamp represents the time each operation was performed. A complete event is considered an activity, and each activity corresponds to one or more operations, such as purchasing an item, which includes adding it to the cart or purchasing it directly, followed by the payment process. Each operation includes the object of the operation and the operation type, such as the object being the "Add to Cart Button" and the operation type being "Click".

[0056] Step 202: Obtain the second code after modifying the first code, set the start time of operation loading according to the timestamp, associate the activity ID, and start the service application.

[0057] The timestamps here apply to all operations. You can set the timestamp corresponding to the first operation as the start time, or you can set a preset timestamp as the start time. The preset timestamp must be no later than all timestamps in the user behavior data.

[0058] Associating the activity ID means linking the activity identifier in the user behavior data with the activity identifier in the second code, so that the operation corresponding to the activity in the user behavior data can be executed in the program after the second code runs.

[0059] Starting the service application refers to enabling the relevant services that run the second code in the test scenario. These relevant services simulate the relevant services running online, so that the second code can run as if it were online.

[0060] The second code is a modified version of the first code.

[0061] Modifications to the first code can include adding new features or optimizing existing features.

[0062] Step 203: Run the second code based on the operation object and operation type to generate the second operation result.

[0063] Typically, after modifying code that runs online offline, backtesting is required. If the verification passes, the code can be deployed online; otherwise, the code needs to be modified again.

[0064] Typically, after modifying code that runs online offline, backtesting is required. If the verification passes, the code can be deployed online; otherwise, the code needs to be modified again.

[0065] Therefore, the method of backtesting is crucial. If the backtesting method is inappropriate, it may not be able to effectively verify whether the modifications to the initial code meet expectations.

[0066] If backtesting is performed using test cases, but these test cases are not real data, the scenarios that can be verified are small, and the amount of data is insufficient, problems in the code may not be detected. Such backtesting will pass, and deploying such code may cause major problems.

[0067] This application does not use test cases for backtesting verification. Instead, it uses real user behavior data from the online runtime for backtesting verification, just like running the second code online. If there are problems with the second code, they can be discovered early.

[0068] Step 204: Compare the results of the first operation and the second operation to verify whether the modification of the first code is correct.

[0069] By analyzing and comparing the results of the first and second operations, such as in a shopping scenario, comparing order volume trends and purchase limits, it can be determined whether the data corresponding to the second operation result is abnormal, and then it can be determined whether the modification of the first code is correct, that is, whether it meets expectations.

[0070] In practice, the correctness of the modification to the first code is determined based on the purpose of the modification:

[0071] If the purpose of modifying the first code is to add a new function, it is necessary to determine whether the new function will affect the original function to determine whether the modification of the first code is correct.

[0072] If the purpose of modifying the first code is to optimize the original function, it is necessary to determine whether the optimized function affects other functions and whether the optimized function can be used normally, in order to determine whether the modification of the first code has achieved the expected optimization function, that is, whether the modification of the first code is correct.

[0073] In this embodiment, user behavior data from the online execution of the first code within a preset time period is used as test data to test the second code, which is a modified version of the first code, in a simulated real user environment. This approach ensures the accuracy of backtesting by testing the offline code with real user behavior data.

[0074] Example 3

[0075] See Figure 3 , Figure 3 This is a schematic diagram of the backtesting verification process in Embodiment 3 of this application. The specific steps are as follows:

[0076] Step 301: Obtain the CSV file corresponding to the first operation result generated by the first code running online within a preset time, as well as the recorded user behavior data.

[0077] When the code runs online, real users will perform user operations, resulting in corresponding operation results.

[0078] If the online operation time is relatively long, a large number of operation results and user behavior data will be generated; in this embodiment of the application, the operation results and user behavior data corresponding to the preset time period are obtained.

[0079] In the existing implementation, if the code has the function of recording user behavior data when running online, it can be recorded when the code runs. If not, this function needs to be configured to record user behavior data.

[0080] To facilitate and visualize the comparison of operation results, the first operation result is stored as a comma-separated values ​​(CSV) file. If the first operation result is not stored as a CSV file when the first code is run online, it can be converted to a CSV file.

[0081] Step 302: Obtain the second code after modifying the first code, run the second code based on the user behavior data, generate the second operation result, and store it in CSV format.

[0082] Modifications to the first code can include adding new features or optimizing existing features.

[0083] Typically, after modifying code that runs online offline, backtesting is required. If the verification passes, the code can be deployed online; otherwise, the code needs to be modified again.

[0084] Therefore, the method of backtesting is crucial. If the backtesting method is inappropriate, it may not be able to effectively verify whether the modifications to the initial code meet expectations.

[0085] If backtesting is performed using test cases, but these test cases are not real data, the scenarios that can be verified are small, and the amount of data is insufficient, problems in the code may not be detected. Such backtesting will pass, and deploying such code may cause major problems.

[0086] This application does not use test cases for backtesting verification. Instead, it uses real user behavior data from the online runtime for backtesting verification, just like running the second code online. If there are problems with the second code, they can be discovered early.

[0087] In this embodiment of the application, when the second code is run, the corresponding second operation result is stored in CSV format.

[0088] Step 303: Read the first operation result and the second operation result in CSV format using R language, compare the first operation result and the second operation result, and verify whether the modification of the first code is correct.

[0089] By analyzing and comparing the results of the first and second operations, such as in a shopping scenario, comparing order volume trends and purchase limits, it can be determined whether the data corresponding to the second operation result is abnormal, and then it can be determined whether the modification of the first code is correct, that is, whether it meets expectations.

[0090] In practice, the correctness of the modification to the first code is determined based on the purpose of the modification:

[0091] If the purpose of modifying the first code is to add a new function, it is necessary to determine whether the new function will affect the original function to determine whether the modification of the first code is correct.

[0092] If the purpose of modifying the first code is to optimize the original function, it is necessary to determine whether the optimized function affects other functions and whether the optimized function can be used normally, in order to determine whether the modification of the first code has achieved the expected optimization function, that is, whether the modification of the first code is correct.

[0093] Reading CSV files in R can be done using either the `read.table` function or the `read.csv` function, which is essentially a wrapper around `read.table` with the `sep` parameter set to a comma (","). The `read.table` function returns a `data.frame`. This embodiment does not impose any restrictions on how to read data from CSV files using R.

[0094] After reading the data from the CSR file using the R language, the data can also be visualized to provide the most intuitive comparison results.

[0095] Taking the scenario of limited purchases for mall users as an example, we use R language to draw the trend chart of daily order volume based on the results of the first and second operations.

[0096] See Figure 4 , Figure 4 This is a schematic diagram illustrating the order volume trend in the embodiments of this application. Figure 4 In the graph, the horizontal axis represents time, and the vertical axis represents the number of orders. Figure 4 The curve corresponding to L1 is the order volume trend chart displayed for the first operation result, and the curve corresponding to L2 is the order volume trend chart displayed for the second operation result.

[0097] from Figure 4 As can be seen, the order volume trends are significantly different at the same time, indicating that the second operation result is abnormal, and further indicating that the modification of the second code is not as expected, that is, there is a problem.

[0098] In this embodiment, user behavior data from the online execution of the first code within a preset time period is used as test data. The second code, a modified version of the first code, is then tested to simulate a real user environment. The operation results are saved as a CSV file, and the CSV file is read using R language for intuitive comparison and analysis of abnormal data. This approach is characterized by comprehensive and realistic test data, intuitive comparison, and rapid anomaly detection. Testing the offline code with real user behavior data ensures the accuracy of backtesting verification.

[0099] Based on the same inventive concept, this application also provides a backtesting verification device. See also Figure 5 , Figure 5 This is a schematic diagram of the backtesting verification device in an embodiment of this application. The device includes: a first acquisition unit 501, a second acquisition unit 502, a running unit 503, and an analysis unit 504;

[0100] The first acquisition unit 501 is used to acquire the first operation result generated by the first code running online within a preset time, as well as the recorded user behavior data;

[0101] The second acquisition unit 502 is used to acquire the second code after the first code has been modified.

[0102] The execution unit 503 is used to run the second code obtained by the second acquisition unit 502 based on the user behavior data obtained by the first acquisition unit 501, and generate the second operation result.

[0103] The analysis unit 504 is used to compare the first operation result obtained by the first acquisition unit 501 with the second operation result generated by the running unit 503 to verify whether the modification of the first code is correct.

[0104] In another embodiment,

[0105] The first acquisition unit 501 acquires user behavior data including: timestamp, activity identifier ID, operation object, and operation type;

[0106] The execution unit 503 is specifically used to run the second code based on the user behavior data, including: setting the start time of operation loading according to the timestamp, associating the activity ID, and starting the service application; and running the second code based on the operation object and operation type.

[0107] In another embodiment,

[0108] The result of the first operation is stored in CSV format;

[0109] The result of the second operation is stored in CSV format.

[0110] In another embodiment,

[0111] The analysis unit 504 is specifically used to compare the first operation result and the second operation result by reading the first operation result in CSV format and the second operation result in CSV format using R language, and then comparing the first operation result and the second operation result.

[0112] The units in the above embodiments can be integrated into one unit or deployed separately; they can be merged into one unit or further divided into multiple sub-units.

[0113] In another embodiment, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the backtesting verification method.

[0114] In another embodiment, a computer-readable storage medium is also provided, on which computer instructions are stored, which, when executed by a processor, can implement the steps in the backtesting verification method.

[0115] Figure 6 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Figure 6 As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640. The processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions stored in the memory 630 to execute the following methods:

[0116] Obtain the first operation result generated by the first code running online within a preset time, as well as the recorded user behavior data;

[0117] Obtain the second code after modifying the first code, run the second code based on the user behavior data, and generate a second operation result;

[0118] Compare the results of the first operation and the second operation to verify whether the modification to the first code is correct.

[0119] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0120] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0121] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0122] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A backtesting verification method, characterized in that, The method includes: Obtain the first operation result generated by the first code running online within a preset time, as well as the recorded user behavior data; wherein, the user behavior data is real user data; Obtain the second code after modifying the first code, run the second code based on the user behavior data, and generate a second operation result; wherein, the user behavior data is used to backtest and verify the second code; Compare the results of the first operation and the second operation to verify whether the modification to the first code is correct.

2. The method according to claim 1, characterized in that, The user behavior data includes: timestamp, activity identifier ID, operation object, and operation type; The execution of the second code based on the user behavior data includes: Set the start time for operation loading based on the timestamp, associate the activity ID, and start the service application; The second code is executed based on the operation object and operation type.

3. The method according to claim 1, characterized in that, The method further includes: Store the result of the first operation in comma-separated CSV format; The result of the second operation is stored in CSV format.

4. The method according to claim 3, characterized in that, The comparison of the first operation result and the second operation result includes: Use R language to read the first operation result and the second operation result in CSV format, and compare the first operation result and the second operation result.

5. A backtesting verification device, characterized in that, The device includes: a first acquisition unit, a second acquisition unit, an operation unit, and an analysis unit; The first acquisition unit is used to acquire the first operation result generated by the first code running online within a preset time, as well as the recorded user behavior data; wherein, the user behavior data is real user data; The second acquisition unit is used to acquire the second code after modifying the first code; The running unit is used to run the second code obtained by the second acquisition unit based on the user behavior data obtained by the first acquisition unit, and generate a second operation result; wherein, the user behavior data is used to backtest and verify the second code; The analysis unit is used to compare the first operation result obtained by the first acquisition unit with the second operation result generated by the running unit to verify whether the modification of the first code is correct.

6. The apparatus according to claim 5, characterized in that, The user behavior data acquired by the first acquisition unit includes: timestamp, activity identifier ID, operation object, and operation type; The running unit, specifically used to run the second code based on the user behavior data, includes: setting the start time of operation loading according to the timestamp, associating the activity ID, and starting the service application; and running the second code based on the operation object and operation type.

7. The apparatus according to claim 5, characterized in that, The result of the first operation is stored in comma-separated value CSV format; The result of the second operation is stored in CSV format.

8. The apparatus according to claim 7, characterized in that, The analysis unit is specifically used to compare the first operation result and the second operation result by reading the first operation result in CSV format and the second operation result in CSV format using R language, and then comparing the first operation result and the second operation result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method described in any one of claims 1-4.

Images