Serverless Fuzz Testing via Application Copying
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Developers of serverless applications face challenges in testing for runtime errors due to the ephemeral nature of serverless compute services, where traditional fuzz testing methods are not directly applicable, as they require persistent computing environments and manual resource management.
Innovation Solution
A system for fuzz testing serverless applications is introduced, utilizing a fuzzer that generates program inputs to test serverless applications, with a monitor application to identify and verify error-causing inputs by executing a copy of the application, allowing continuous testing without interruption and leveraging logging services for error analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional fuzz testing methods are used, then testing can be performed on persistent computing environments, but the ephemeral nature of serverless compute services makes these methods inapplicable
Solution Approach 1:
The system creates a copy of the serverless application instance specifically for fuzz testing purposes. This copy allows traditional fuzz testing methods to be applied without affecting the original application's ephemeral nature, as the fuzzer operates on the replicated instance while the original continues its normal lifecycle.
Solution Approach 2:
The system introduces an intermediary layer that bridges traditional fuzz testing methods and serverless compute services. This intermediary manages the creation, coordination, and teardown of application copies, allowing fuzz testing to occur without requiring direct modification of the serverless infrastructure or the original application instance.
2Reliability
If the original application is stopped for testing, then fuzz testing can be performed, but continuous execution and testing cannot be maintained
Solution Approach 1:
The system segments the application into two independent instances: the original application instance that continues to execute normally and produces value, and a separate copy instance dedicated to fuzz testing. This segmentation allows both continuous execution of the original application and comprehensive testing of the copy to occur simultaneously without interference.
Solution Approach 2:
A copy of the serverless application is created specifically for testing purposes. This copy can be stopped, tested, and restarted without affecting the original application's continuous execution, enabling fuzz testing to proceed while the original application maintains its productivity and service availability.
3Ease of operation
If manual resource management is required, then persistent environments can be controlled, but automated serverless resource management becomes complex
Solution Approach 1:
The system employs self-service mechanisms where the serverless compute service automatically manages the lifecycle of application copies. The service handles instantiation, configuration, and teardown of test instances based on triggers from the fuzz testing system, eliminating the need for manual resource management while maintaining full control over the testing process.
Solution Approach 2:
An intermediary management layer is introduced that sits between the fuzz testing system and the serverless compute service. This intermediary automatically handles resource provisioning, coordination, and cleanup, translating testing requirements into serverless resource operations without requiring manual intervention or complex resource management from users.
Data Source
AI summary
Techniques are described for “fuzz testing” serverless software applications (for example, software applications implemented by a serverless compute service provided by a service provider network). Fuzz testing is an automated testing technique involving a fuzzer client application (or simply “fuzzer”) providing invalid, unexpected, or random data as inputs to a software application that is the target of the testing to discover software bugs and other runtime issues during execution of the software. To fuzz test a serverless application, a fuzzer continuously sends inputs to the serverless application and, when a runtime error is detected (for example, by monitoring log data generated by the serverless application), a copy of the serverless application can be invoked with the suspected error-inducing input to verify that the input indeed causes the error without interrupting the fuzzer's testing of the original serverless application.