Shadow testing system and shadow testing device

Abstract

A shadow testing system (1000) comprises a test case collection unit (21) that collects a request to a current service system (100) and a response that is a processing result of the current service system (100) corresponding to the request, as a request-response set, and extracts a request-response set that does not overlap with predetermined service feature information that characterizes processing content regarding a request to the current service system (100). A shadow testing device (30) is provided with a shadow testing execution unit (32) that inputs the request from the extracted request-response set to a new service system (150), and acquires a response to the request.

Description

Shadow test system and shadow test equipment

[0001] The present invention relates to a shadow test system and a shadow test device for performing a shadow test on an old service system and a new service system.

[0002] In the case of web services on a network, various test methods have been developed to check whether an application operates normally when a new version of the application is deployed (see Non-Patent Document 1). Typical methods include a deployment creation pattern, a rolling update, a canary test, and a shadow test.

[0003] Among these methods, the shadow test mirrors (copies) real traffic and inputs it to both the new version and the old version to compare their behavior (see Non-Patent Document 2). For example, as shown in Fig. 7, the load balancer 5 transmits traffic acquired from the user terminal 1 to the old service system 100a as real traffic, and also transmits the mirrored traffic to the new service system 150a.

[0004] This shadow testing has the following advantages: (1) It is possible to test using real traffic. In other words, there is no need to prepare test cases. (2) There is an old (existing) service as a clear standard of operation, making it easy to compare with the new service. (3) Because the traffic is replicated, bugs that occur in the new service system that processes the shadow data do not affect the production environment (old service system). (4) Because two environments, one for the old service system and one for the new service system, are prepared, there is zero downtime (stoppage or interruption) or time required for rollback.

[0005] Furthermore, since the new version of the service system has no effect on the old (existing) version of the service system, it is also possible to test the new version offline. As shown in Figure 8, first, data input and output to and from the old service system 100a while online is recorded. Then, while offline, the input data is input to the new service system 150a to obtain output information, and the old service and the new service are compared.

[0006] Shadow testing, including offline testing, is characterized by the following: (1) using real traffic as a test case, and (2) evaluating by comparing the behavior of the old and new versions.

[0007] Google Cloud, "Application Deployment and Testing Strategies: Shadow Testing Patterns," [online], [Retrieved March 22, 2024], Internet <URL: https: / / cloud.google.com / architecture / application-deployment-and-testing-strategies?hl=ja> Code With Engineering Playbook, "Shadow Testing," [online], [Retrieved March 22, 2024], Internet <URL: https: / / microsoft.github.io / code-with-engineering-playbook / automated-testing / shadow-testing / >

[0008] However, shadow testing has the following challenges: While the real traffic used in shadow testing is realistic, when viewed as test cases, there is a lot of similar traffic and overlap, making it time-consuming to run. Also, shadow testing requires running both the old and new services, which increases the cost of running the test.

[0009] The present invention has been made in view of the above points, and an object of the present invention is to shorten the test execution time and reduce the execution cost in shadow testing.

[0010] The shadow test system of the present invention is a shadow test system comprising a shadow test device that performs a shadow test comparing an old service system that executes a service using an existing version with a new service system that executes a service using a new version, and is characterized in that the shadow test system comprises a test case collection unit that collects requests to the old service system and responses that are the processing results of the old service system corresponding to the requests as request-response pairs, and extracts as test cases the request-response pairs that do not overlap with specified service characteristic information that characterizes the processing content of the requests to the old service system, and the shadow test device comprises a shadow test execution unit that inputs the extracted request-response pairs to the new service system and obtains the responses to the requests.

[0011] According to the present invention, in a shadow test, it is possible to reduce the test execution time and execution costs.

[0012] FIG. 1 is a diagram showing the overall configuration of a shadow test system according to a first embodiment of the present invention. FIG. 2 is a diagram showing the overall configuration of a shadow test system according to a second embodiment of the present invention. FIG. 3 is a diagram showing the overall configuration of a shadow test system according to a first modified example of the first embodiment of the present invention. FIG. 4 is a diagram showing the overall configuration of a shadow test system according to a second modified example of the first embodiment of the present invention. FIG. 5 is a diagram showing the overall configuration of a shadow test system according to a third modified example of the first embodiment of the present invention. FIG. 6 is a hardware configuration diagram showing an example of a computer that realizes the functions of a proxy server and a shadow test device according to the present embodiment. FIG. 7 is a diagram for explaining an overview of a conventional shadow test. FIG. 8 is a diagram for explaining cases where a shadow test is performed online and offline.

[0013] Next, an embodiment of the present invention (hereinafter referred to as "the present embodiment") will be described. First, an outline of the shadow test system according to the present embodiment will be described.

[0014] The shadow testing system of this embodiment pre-configures information (predetermined service characteristic information) that characterizes the output (response) and processing content of the old service system in response to the input (request) to the old service system. When collecting actual input data as test cases for shadow testing, the shadow testing system does not record data identical to the predetermined service characteristic information, but records data different from the predetermined service characteristic information as a unique test case. Note that a test case refers to a combination of execution conditions, input data, expected output, and results, etc., prepared when conducting software testing. This allows the shadow testing system to reduce duplicate test cases, thereby reducing execution costs and time.

[0015] The shadow test system uses, for example, (1) input-output pairs in the old software executed by the old service system, and (2) a set of function calls for inputs from the old software, as parameters for information (predetermined service characteristic information) that characterizes the output (response) and processing content in response to an input (request) to the old service system. Specific examples of this will be described below as the first and second embodiments. When recording and using real input data as test cases, the shadow test system can execute shadow tests using unique test cases by excluding test cases that are determined to overlap with this predetermined service characteristic information. This will be described in detail below.

[0016] First Embodiment First, a shadow test system 1000 (1000A) according to a first embodiment of the present invention will be described. The shadow test system 1000 (1000A) uses input-output pairs from the old software executed by the old service system as predetermined service feature information. Specifically, the shadow test system 1000 (1000A) defines, as predetermined service feature information, classification rules (equivalence relations) for pairs of input data and output data that can be considered identical by the user in advance for a web service, such as request-response. Then, when recording test cases online, the shadow test system 1000 (1000A) avoids recording identical request-responses based on the defined classification rules (equivalence relations). This allows the shadow test system 1000 (1000A) to reduce the number of test cases, thereby reducing execution costs and test time. The execution cost includes the actual operating time of the server's CPU, processing volume, etc., and in cases where the amount of resources used is a monetary cost, such as in the cloud, the cost can be reduced by reducing the number of test cases.

[0017] FIG. 1 is a diagram showing the overall configuration of a shadow test system 1000 (1000A) according to a first embodiment of the present invention. As shown in FIG. 1, in the shadow test system 1000 (1000A), a proxy server 20 is provided between a user terminal 1 and an old service system 100 that accepts requests from the user terminal 1 and provides Web services. The shadow test system 1000 (1000A) includes the proxy server 20 and a shadow test device 30 that is connected to the proxy server 20 via communication. Using test cases collected by the proxy server 20, the shadow test device 30 executes a shadow test of the connected new service system 150.

[0018] The old service system 100 is a system that executes services using old (existing version) software. It provides services by receiving requests from the user terminal 1, executing processing according to the requests, and returning responses to the user terminal 1. This old service system 100 is composed of, for example, one or more servers.

[0019] The new service system 150 is a system that executes services using new (new version) software. When the old service system 100 is switched to the new service system 150, the shadow test device 30 tests whether the service can be provided normally without any abnormalities. The new service system 150 is configured, for example, by one or more servers.

[0020] The proxy server 20 is a server that relays information between the user terminal 1 and the old service system 100. This proxy server 20 transfers requests from the user terminal 1 to the old service system 100, and also transfers responses from the old service system 100 to the user terminal 1 (steps S1 to S4 in FIG. 1). The proxy server 20 includes a test case collection unit 21.

[0021] The test case collection unit 21 collects pairs of requests received from the user terminal 1 and responses (request-response pairs) sent in response to those requests by the old service system 100. The test case collection unit 21 then extracts unique request-response pairs based on classification rules (equivalence relations) relating to requests and responses that are set in advance as predetermined service feature information.

[0022] In detail, if the test case collection unit 21 determines, based on the classification rules, that a request-response pair collected from the old service system 100 is equivalent to a preset request-response pair, the test case collection unit 21 does not extract the request-response pair. On the other hand, if the test case collection unit 21 determines, based on the classification rules, that the request-response pair is not equivalent to a preset request-response pair, the test case collection unit 21 extracts the request-response pair as a unique request-response pair.

[0023] Specifically, the determination of equivalence based on this classification rule is performed based on the following criteria: - If the values ​​of preset fields among the HTTP request-response fields are classified as matching, they are determined to be equivalent. - If the protocols of the L1-L4 headers and some of the values ​​of preset fields among the fields are classified as matching, they are determined to be equivalent.

[0024] More specifically, the following examples of equivalence relations may be set: - If the HTTP request method and URL, and further the response, are classified as matching, it is determined to be equivalent. - If the 5-tuple (source IP, source port number, destination IP, destination port number, protocol number) is classified as matching, it is determined to be equivalent.

[0025] The test case collection unit 21 is configured with classification rules for determining equivalence relations, which are determined to achieve classification with an appropriate granularity depending on the content and characteristics of the service. The test case collection unit 21 transmits information on unique request-response pairs extracted from the actual traffic to the shadow test device 30 (step S5).

[0026] The shadow test device 30 executes a shadow test of the new service system 150 using test cases acquired from the test case collection unit 21. This shadow test device 30 is realized by a computer comprising a control unit, a storage unit, and an input / output unit (not shown). The control unit is configured to include a test case acquisition unit 31 and a shadow test execution unit 32. The storage unit also stores a test case DB (Data Base) 300 that records the acquired test cases.

[0027] The test case acquisition unit 31 acquires unique request-response pairs collected by the test case collection unit 21, and stores them in the test case DB 300 as test cases for shadow testing.

[0028] The shadow test execution unit 32 executes a shadow test using a unique request-response pair stored in the test case DB 300. The shadow test execution unit 32 can execute a shadow test online or offline.

[0029] When executing a shadow test online, the shadow test execution unit 32 acquires in real time information about unique request-response pairs acquired by the test case acquisition unit 31 and recorded in the test case DB 300, and transmits the request information to the new service system 150 (step S6).The shadow test execution unit 32 then acquires a response to the request from the new service system 150 (step S7) and compares it with the response information in the request-response pair recorded in the test case DB 300.

[0030] When executing a shadow test offline, the shadow test execution unit 32 acquires, at any timing, information on unique request-response pairs acquired by the test case acquisition unit 31 and recorded in the test case DB 300, and transmits the request information to the new service system 150. Note that in the offline case, it is not necessary to take the same test time as for real traffic, and the request information can be transmitted sequentially to the new service system 150 in a compressed time. Therefore, in the offline case, in addition to the effect of reducing execution costs due to the reduction in test cases, it is also possible to shorten the test time.

[0031] Second Embodiment Next, a shadow test system 1000 (1000B) according to a second embodiment of the present invention will be described. The shadow test system 1000 (1000B) uses a set of function calls for input from the old software as a parameter of predetermined service characteristic information. For example, the set of functions called by the server of the old service system that received a request to process the request, as well as the arguments and return values ​​of those functions, are defined as predetermined service characteristic information. The test case collection unit 21 (21B) of the proxy server 20 acquires information on the set of function calls for processing the request from the old service system 100, along with the response to the request. The test case collection unit 21B then determines whether the set of function calls is identical. In other words, the test case collection unit 21 (21B) determines whether the set of predefined functions matches the set of functions received with the response to the request (the "function information" described below). If they match, the test case collection unit 21 (21B) does not extract (record) the set of function calls as a test case. On the other hand, the test case collection unit 21 (21B) extracts (records) unique request-response pairs that do not match, thereby enabling the shadow test system 1000 (1000B) to reduce execution costs and test time.

[0032] 2 is a diagram showing the overall configuration of a shadow test system 1000 (1000B) according to a second embodiment of the present invention. Functions that perform the same processing as those in the shadow test system 1000 (1000A) shown in FIG. 1 are given the same names and symbols, and descriptions thereof will be omitted. As shown in FIG. 2, the shadow test system 1000 (1000B) differs from the first embodiment in that the old service system 100 (100B) includes a function information extraction unit 11 and the proxy server 20 includes a test case collection unit 21 (21B).

[0033] Each server in the old service system 100 (100B) is equipped with a function information extraction unit 11. This function information extraction unit 11 monitors the functions (sets of functions) that a server that receives a request calls when processing the request, and extracts them as function information. Depending on the granularity of the identity determination rules defined in the test case collection unit 21B (described later), the function information extraction unit 11 may obtain information on a sequence of function calls as information with a higher granularity than a set of functions, or may extract information on the arguments and return values ​​of the functions as function information.

[0034] Each server in the old service system 100 (100B) transmits a response to the received request, including the function information extracted by the function information extraction unit 11, to the proxy server 20 (step S3B in FIG. 2).

[0035] The test case collection unit 21 (21B) collects pairs of requests received from the user terminal 1 and responses (request-response pairs) sent in response to those requests by the old service system 100. The test case collection unit 21B then extracts unique request-response pairs based on identity rules for sets of function calls that are set in advance as predetermined service feature information.

[0036] In detail, if the test case collection unit 21B determines that a set of function calls preset in the identity rule for sets of function calls is identical to the acquired function information, it does not extract a request-response pair for that function information. On the other hand, if the test case collection unit 21B determines that a set of function calls preset is not identical to the acquired function information, it extracts a request-response pair for that function information as a unique request-response pair.

[0037] The granularity of the identity rules for this set of function calls is set according to the service being tested. For example, the set of functions to monitor calls can be set to only the set of system call functions, which are part of the kernel functions, or more specifically, to monitor whether the sequence of function calls (the order of function calls) matches, or whether the arguments and return values ​​match. The granularity is set according to the service.

[0038] To determine whether this set of function calls is the same, the following identity rules are set, for example: ・Rules in which the monitoring target is a system call and the sequence of function calls matches (arguments are not distinguished) (execv → brk → mmap → faccessat → openat → ...) ・Example of monitoring limited to file access and distinguishing between opened files (openat(" / etc / ...") → close → openat(" / lib / ...") → read → ...)

[0039] If the test case collection unit 21 (21B) determines that the function call set does not match the predetermined identity rule in determining the identity of the function call set, it sends the request-response pair related to the function information to the shadow test device 30 as a unique request-response pair (step S5).

[0040] The processing performed by the shadow test device 30 is the same as that described in FIG. 1 , and therefore a detailed description thereof will be omitted. The shadow test execution unit 32 of the shadow test device 30 executes the shadow test offline. In this case, the shadow test execution unit 32 can execute the shadow test using a reduced number of test cases, thereby reducing execution costs. Furthermore, when the test is executed offline, the test time can be further shortened.

[0041] <Variation 1 of First Embodiment> Next, a shadow test system 1000 (1000C) according to Variation 1 of the first embodiment will be described. FIG. 3 is a diagram showing the overall configuration of the shadow test system 1000 (1000C) according to Variation 1 of the first embodiment of the present invention. Functions that perform the same processing as those in the shadow test system 1000 (1000A) shown in FIG. 1 are given the same names and symbols, and descriptions thereof will be omitted. The shadow test system 1000 (1000C) shown in FIG. 3 differs from the shadow test system 1000 (1000A) shown in FIG. 1 in that the proxy server 20 does not include a test case collection unit 21 but instead includes a data duplication unit 22, and that the shadow test device 30 in FIG. 1 becomes the shadow test device 30 (30C) in FIG. 3 and includes a test case collection unit 21 (21C) instead of the test case acquisition unit 31.

[0042] In the shadow test system 1000 (1000C), the data replication unit 22 of the proxy server 20 sends all collected request-response pairs to the shadow test device 30 (30C). The test case collection unit 21 (21C) of the shadow test device 30 (30C) then extracts unique request-response pairs based on classification rules (equivalence relations) related to requests and responses that are preset as predetermined service characteristic information. In other words, the shadow test system 1000 (1000C) is characterized in that the shadow test device 30 is equipped with a test case collection unit 21 (21C). Differences from the shadow test system 1000 (1000A) are described in detail below.

[0043] The data duplication unit 22 of the proxy server 20 collects (duplicates) a pair of a request received from the user terminal 1 and a response (a request-response pair) sent in response to the request by the old service system 100. The data duplication unit 22 then transmits information about the duplicated request-response pair to the shadow test device 30 (30C) (step S5C in FIG. 3).

[0044] When the test case collection unit 21 (21C) of the shadow test device 30 (30C) acquires information about request-response pairs from the proxy server 20, it extracts unique request-response pairs based on classification rules (equivalence relations) related to requests and responses that are preset as predetermined service feature information.The test case collection unit 21C then stores the extracted unique request-response pairs in the test case DB 300 as test cases for the shadow test.

[0045] This also allows the shadow test device 30 (30C) to execute shadow tests using unique request-response pairs stored in the test case DB 300. Therefore, the shadow test device 30 (30C) of the shadow test system 1000 (1000C) can execute shadow tests using a reduced number of test cases, thereby reducing execution costs. Furthermore, when executing offline, it is possible to further shorten the test time.

[0046] In the shadow test system 1000 (1000C), each server of the old service system 100 may be provided with a function information extraction unit 11, as in the shadow test system 1000 (1000B) shown in FIG. 2. In this case, the function information extraction unit 11 monitors the set of functions called in response to requests and transmits the function information to the proxy server 20 along with the response to the received request. The data replication unit 22 of the proxy server 20 then transmits information in which the function information is attached to the request-response pair to the shadow test device 30 (30C). The test case collection unit 21 (21C) of the shadow test device 30 (30C) then extracts unique request-response pairs based on identity rules for sets of function calls that are preset as predetermined service characteristic information, similar to the test case collection unit 21 (21B) of FIG. 2. This also allows the shadow test device 30 (30C) to perform shadow testing using unique request-response pairs stored in the test case DB 300.

[0047] <Variation 2 of First Embodiment> Next, a shadow test system 1000 (1000D) according to Variation 2 of the first embodiment will be described. FIG. 4 is a diagram showing the overall configuration of the shadow test system 1000 (1000D) according to Variation 2 of the first embodiment of the present invention. Functions that perform the same processing as those in the shadow test system 1000 (1000A) shown in FIG. 1 are given the same names and symbols, and descriptions thereof will be omitted. The shadow test system 1000 (1000D) shown in FIG. 4 differs from the shadow test system 1000 (1000A) shown in FIG. 1 in that it does not include a proxy server 20, and that each server of the old service system 100 (100D) includes a test case collection unit 21 (21D) instead of the proxy server 20.

[0048] In the shadow test system 1000 (1000D), each server of the old service system 100 is provided with a test case collection unit 21 (21D). The test case collection unit 21D extracts unique request-response pairs based on classification rules (equivalence relations) related to requests and responses that are preset as predetermined service feature information, and transmits them to the shadow test device 30. In other words, each server of the old service system 100 (100D) is provided with a test case collection unit 21D. The differences from the shadow test system 1000 (1000A) will be described in detail below.

[0049] The test case collection unit 21 (21D) provided in each server of the old service system 100 (100D) collects pairs of requests received from the user terminal 1 and responses resulting from the execution of processing in response to those requests (request-response pairs).The test case collection unit 21D then extracts unique request-response pairs based on classification rules (equivalence relations) related to requests and responses that are preset as predetermined service feature information.The test case collection unit 21D transmits information on the extracted unique request-response pairs to the shadow test device 30 (step S5D in FIG. 10).

[0050] The CPU (control unit) of each server in the old service system 100 (100D) can be equipped with the functions of the test case collection unit 21 (21D), which can be achieved, for example, by using an eBPF (Extended Berkeley Packet Filter) or a service mesh (sidecar) in a container environment.

[0051] The test case acquisition unit 31 of the shadow test device 30 acquires unique request-response pairs collected by the test case collection unit 21 (21D) and stores them as test cases for the shadow test in the test case DB 300. The shadow test execution unit 32 then executes the shadow test using the unique request-response pairs, which are the test cases stored in the test case DB 300.

[0052] This also allows the shadow test system 1000 (1000D) to execute shadow tests using unique request-response pairs. The shadow test device 30 of the shadow test system 1000 (1000D) can execute shadow tests using a reduced number of test cases, thereby reducing execution costs. Furthermore, when executing offline, it is possible to further shorten the test time.

[0053] In the shadow test system 1000 (1000D), each server of the old service system 100 (100D) may further include a function information extraction unit 11, as in the shadow test system 1000 (1000B) shown in FIG. 2. In this case, the function information extraction unit 11 monitors the set of functions called in response to requests and outputs the extracted function information to the test case collection unit 21D. Then, similar to the test case collection unit 21B of FIG. 2, the test case collection unit 21D of the old service system 100 (100D) extracts unique request-response pairs based on identity rules for sets of function calls that are preset as predetermined service characteristic information. This also allows the shadow test device 30 of the shadow test system 1000 (1000D) to perform shadow testing using unique request-response pairs stored in the test case DB 300.

[0054] <Variation 3 of the First Embodiment> Next, a shadow test system 1000 (1000E) according to Variation 3 of the first embodiment will be described. FIG. 5 is a diagram showing the overall configuration of the shadow test system 1000 (1000E) according to Variation 3 of the first embodiment of the present invention. Functions that perform the same processing as those in the shadow test system 1000 (1000A) shown in FIG. 1 are given the same names and symbols, and descriptions thereof will be omitted. As shown in FIG. 5, the shadow test system 1000 (1000E) differs from the first embodiment in that the old service system 100 (100E) includes a behavior information extraction unit 12 and the proxy server 20 (20E) includes a behavior information collection unit 23. Furthermore, the shadow test device 30 (30E) differs from the shadow test device 30 of FIG. 1 in that it includes a behavior information acquisition unit 33 and a behavior information verification unit 34. Another difference is that the new service system 150 (150E) includes a behavior information extraction unit 12.

[0055] Each server in the old service system 100 (100E) is equipped with a behavior information extraction unit 12. This behavior information extraction unit 12 extracts information (behavior information) for comparing the processing contents and processing results (behavior) between the old service system 100 (100E) and the new service system 150 (150E). This behavior information includes, for example, response time, CPU time used, memory usage, and service-specific execution logs, and is information obtained by executing a service.

[0056] Each server of the old service system 100 (100E) transmits the behavior information extracted by the behavior information extraction unit 12 to the proxy server 20 (20E) in addition to a response to the received request (step S3E in FIG. 5).

[0057] The test case collection unit 21 of the proxy server 20 (20E) extracts unique request-response pairs and sends them to the shadow test device 30. In addition, the behavior information collection unit 23 transfers the behavior information acquired from the old service system 100 (100E) to the shadow test device 30E (step S5E).

[0058] The behavior information acquisition unit 33 of the shadow test device 30 (30E) stores the behavior information acquired from the proxy server 20 (20E). The shadow test execution unit 32 also sends a request to the new service system 150 (150E) (step S6). Each server in the new service system 150 (150E) then sends the behavior information extracted by the behavior information extraction unit 12 to the shadow test device 30 (30E) in addition to a response to the received request (step S7E in FIG. 5). The behavior information verification unit 34 acquires the behavior information extracted by the behavior information extraction unit 12 of the new service system 150 (150E). The behavior information verification unit 34 then compares the behavior information in the old service system 100 (100E) with the behavior information in the new service system 150 (150E) to verify the old and new service systems.

[0059] In this way, the shadow test system 1000 (1000E) can execute the shadow test using a reduced number of test cases. Also, by comparing the behavior information of the old and new service systems, the shadow test system 1000 (1000E) can more accurately verify whether there are any problems with the deployment of the new service system.

[0060] <Hardware Configuration> The proxy server 20 (20E) and the shadow test device 30 (30C, 30E) according to this embodiment are realized, for example, by a computer 900 configured as shown in Fig. 6. Fig. 6 is a hardware configuration diagram showing an example of the computer 900 that realizes the functions of the proxy server 20 (20E) and the shadow test device 30 (30C, 30E) according to this embodiment. The computer 900 has a CPU 901, a ROM (Read Only Memory) 902, a RAM 903, an HDD (Hard Disk Drive) 904, an input / output I / F (Interface) 905, a communication I / F 906, and a media I / F 907.

[0061] The CPU 901 operates based on programs stored in the ROM 902 or the HDD 904, and performs control by each control unit. The ROM 902 stores a boot program executed by the CPU 901 when the computer 900 is started up, programs related to the hardware of the computer 900, and the like.

[0062] The CPU 901 controls an input device 910 such as a mouse or keyboard, and an output device 911 such as a display or printer, via an input / output I / F 905. The CPU 901 acquires data from the input device 910 via the input / output I / F 905, and outputs generated data to the output device 911. Note that a GPU (Graphics Processing Unit) or the like may be used as a processor together with the CPU 901.

[0063] The HDD 904 stores programs executed by the CPU 901 and data used by the programs. The communication I / F 906 receives data from other devices via a communication network (e.g., NW (Network) 920) and outputs the data to the CPU 901, and also transmits data generated by the CPU 901 to other devices via the communication network.

[0064] The media I / F 907 reads a program or data stored in the recording medium 912 and outputs it to the CPU 901 via the RAM 903. The CPU 901 loads a program related to a target process from the recording medium 912 onto the RAM 903 via the media I / F 907, and executes the loaded program. The recording medium 912 is an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto Optical Disc), a magnetic recording medium, a semiconductor memory, or the like.

[0065] For example, when the computer 900 functions as the proxy server 20 (20E) or the shadow test device 30 (30C, 30E) of the present invention, the CPU 901 of the computer 900 executes a program loaded onto the RAM 903 to realize the functions of each control unit. In addition, the HDD 904 stores data in the RAM 903. The CPU 901 reads and executes a program related to a target process from the recording medium 912. Alternatively, the CPU 901 may read a program related to a target process from another device via a communication network (NW 920).

[0066] <Effects> The following describes the effects of the shadow test system etc. according to the present invention. The shadow test system according to the present invention is a shadow test system 1000 (1000A, 1000B, 1000C, 1000D) including a shadow test device 30 that performs a shadow test comparing an old service system 100 that executes a service according to an existing version with a new service system 150 that executes a service of a new version, wherein the shadow test system 100 is equipped with a test case collection unit 21 (21B, 21C, 21D) that collects requests to the old service system 100 and responses that are the processing results of the old service system 100 corresponding to the requests as request-response pairs, and extracts, as test cases, request-response pairs that do not overlap with predetermined service characteristic information that characterizes the processing content of the requests to the old service system 100, and the shadow test device is equipped with a shadow test execution unit 32 that inputs the requests of the extracted request-response pairs to the new service system 150 and acquires the responses to the requests.

[0067] In this way, the shadow test system 1000 can reduce test cases that overlap with predetermined service feature information, thereby reducing the cost and time required to execute shadow tests by using unique test cases.

[0068] In addition, in the shadow test system 1000, the specified service characteristic information is set using classification rules for request-response pairs, and the test case collection unit 21 extracts, as test cases, request-response pairs that are determined not to be in an equivalence relationship based on the classification rules.

[0069] In this way, the shadow test system 1000 can extract request-response pairs that are determined not to have an equivalence relationship based on the classification rules for request-response pairs and execute the shadow test. Therefore, the shadow test system 1000 can reduce the cost and time of executing the shadow test by using unique (specific) test cases.

[0070] In addition, in the shadow test system 1000 (1000B), the old service system 100 (100B) is equipped with a function information extraction unit 11 that monitors functions called in response to requests and extracts them as function information, and the specified service characteristic information is set as a set of function calls in the old service system 100 (100B) in response to the request, and the test case collection unit 21 (21B) acquires the function information extracted in the old service system 100 (100B) in response to a request for a request-response pair, and if it determines that the set set of function calls and the acquired function information are not identical, it extracts the request-response pair as a test case.

[0071] In this way, the shadow test system 1000 can execute a shadow test using a set of request-response pairs that are determined to be different from the set of function calls and the function information extracted in the old service system. Therefore, the shadow test system 1000 can reduce the cost and time of executing a shadow test by using unique test cases.

[0072] The present invention is not limited to the above-described embodiments, and many modifications can be made by a person skilled in the art within the technical concept of the present invention.

[0073] REFERENCE SIGNS LIST 1 User terminal 11 Function information extraction unit 12 Behavior information extraction unit 20, 20E Proxy server 21, 21B, 21C, 21D Test case collection unit 22 Data replication unit 23 Behavior information collection unit 30, 30C, 30E Shadow test device 31 Test case acquisition unit 32 Shadow test execution unit 33 Behavior information acquisition unit 34 Behavior information verification unit 100, 100B, 100E Old service system 150, 150E New service system 300 Test case DB 1000, 1000B, 1000C, 1000D, 1000E Shadow test system

Claims

1. A shadow test system comprising a shadow test device that executes a shadow test comparing an old service system that executes a service according to an existing version with a new service system that executes a service of a new version, wherein the shadow test system comprises a test case collection unit that collects requests to the old service system and responses that are the processing results of the old service system corresponding to the requests as request-response pairs, and extracts, as test cases, the request-response pairs that do not overlap with predetermined service characteristic information that characterizes the processing content of the requests to the old service system, and the shadow test device comprises a shadow test execution unit that inputs the extracted request-response pairs to the new service system and obtains the responses to the requests.

2. The shadow test system of claim 1, wherein the predetermined service characteristic information is set by a classification rule for the request-response pairs, and the test case collection unit extracts, as the test cases, request-response pairs that are determined not to be in an equivalence relationship based on the classification rule.

3. The shadow test system of claim 1, wherein the old service system is equipped with a function information extraction unit that monitors functions called in response to the request and extracts them as function information, the predetermined service characteristic information is set as a set of function calls in the old service system for the request, and the test case collection unit acquires the function information extracted in the old service system in response to the request of the request-response pair, and if it determines that the set set of function calls and the acquired function information are not identical, extracts the request-response pair as the test case.

4. A shadow test device that performs a shadow test to compare an old service system that executes a service based on an existing version with a new service system that executes a service based on a new version, comprising: a test case collection unit that collects requests to the old service system and responses that are the processing results of the old service system corresponding to the requests as request-response pairs, and extracts, as test cases, the request-response pairs that do not overlap with predetermined service characteristic information that characterizes the processing content of the requests to the old service system; and a shadow test execution unit that inputs the extracted request-response pairs to the new service system and obtains the responses to the requests.

Images