Pre-Cached Code Execution for Faster Builds and Fewer Downloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The conventional method of downloading code files from a code hosting platform for compilation, building, and other code processing tasks is inefficient, particularly for large software products, leading to prolonged download times and low completion efficiency due to the need for multiple executors to simultaneously initiate downloads, and repeated downloads when execution tasks overlap.
Innovation Solution
A code execution method and system that includes a pre-preparation service module to download updated code files to a storage cluster based on user-defined policies, allowing direct access to code content from the storage cluster upon request, and an executor that filters out duplicate storage addresses to avoid repeated reading.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple executors simultaneously download code files from a code hosting platform, then each executor can obtain the required code version, but the download time increases significantly and compilation efficiency decreases
Solution Approach 1:
The system performs preliminary actions by downloading and caching code files to the storage cluster in advance based on pre-preparation policies (such as downloading code for upcoming releases or frequently accessed versions). This allows executors to obtain code quickly from the local storage cluster without simultaneous downloads from the remote hosting platform, resolving the contradiction between code availability and download time
Solution Approach 2:
The storage cluster acts as an intermediary between the code hosting platform and executors. Instead of multiple executors simultaneously accessing the remote code hosting platform (causing download conflicts and delays), they access the local storage cluster which has pre-cached the code. This intermediary layer eliminates the simultaneous download problem while maintaining code availability
2Adaptability or versatility
If executors download code files independently without coordination, then each executor can access the required code version, but repeated downloads occur when execution tasks overlap, reducing overall efficiency
Solution Approach 1:
The system merges the code retrieval operations of multiple executors by having them all access the shared storage cluster rather than independently downloading from the remote platform. The pre-preparation service consolidates download requests and caches code files once, making them available to all executors who need them, thus eliminating redundant downloads while maintaining execution flexibility
Solution Approach 2:
The storage cluster provides a universal service to all executors, serving as a common code repository that any executor can access. This multi-functional approach allows the system to support multiple execution tasks simultaneously without requiring each executor to independently download code, thereby improving compilation efficiency while maintaining adaptability
Data Source
Figure 1~2
Figure 3
Figure 4~5
AI summary
This application provides a code execution method and a device. The method includes: A pre-preparation service module obtains a pre-preparation policy set by a user; the pre-preparation service module downloads, based on the pre-preparation policy, an updated code file included in an updated code version from a code hosting platform to a storage cluster; the pre-preparation service module receives a code execution request, and determines, based on the code execution request, a storage address of code content corresponding to the code execution request in the storage cluster; the pre-preparation service module sends the storage address to an executor; the executor obtains the code content based on the received storage address; and the executor executes the obtained code content. This can avoid a problem of low compilation and building completion efficiency caused by a relatively long time occupied by a download action in the conventional technology, and can further resolve a problem in the conventional technology that a code file is repeatedly downloaded when execution tasks received by an executor overlap.