Military and civilian field-oriented complex operation environment rapid construction system
By rapidly building a system using a self-learning Python environment, the problems of low efficiency in building traditional Python environments and high maintenance costs of existing technologies are solved. It enables rapid deployment in complex and isolated network environments, improving environment configuration efficiency and self-learning capabilities.
Patent Information
- Application Number
- CN202511089962.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-05
- Publication Date
- 2025-11-21
AI Technical Summary
Traditional Python environment building methods are inefficient and unstable, especially in complex projects or isolated network environments where rapid deployment is difficult. Furthermore, the existing technology ReadPyE has high maintenance costs, cannot be self-updated, and lacks adaptability, resulting in excessively long deployment cycles for high-precision technologies.
By employing the preprocessing installation and verification module PEMS, the memory retrieval and parsing module MARS, and the automated experience extraction module AIDE, a rapid Python environment building system with self-learning capabilities is constructed. Through a private PyPI mirror source and large model service, intelligent diagnosis and self-learning are achieved, reducing environment building time and costs.
It significantly reduces environment build time in network-isolated environments, from several days to tens of minutes, improving environment configuration efficiency and meeting the rapid deployment needs of military research and a wide range of Python projects.
Smart Images
Figure CN120994440A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of computer engineering, and particularly relates to a complex operation environment rapid construction system for military and civilian fields. BACKGROUND
[0002] In the era of high informationization, high technologies such as big data, block chain and artificial intelligence are deeply promoting the process of military modernization, and the implementation of these technologies often highly depends on an accurate and complex operation environment. As the basic guarantee for the landing of technologies, the construction efficiency and accuracy of the operation environment directly determine whether the high technologies can be successfully deployed, and is a key factor for the high technologies to play their effectiveness. The traditional Python environment construction method relies on manual configuration, and often faces problems such as low efficiency, poor stability and easy errors, especially in complex projects or isolated network environments.
[0003] The current environment configuration work in technology deployment still relies on manual construction by engineers, which has significant drawbacks: Low success rate: engineers need to spend a lot of time combing past experience, selecting available content from complex network information, and then gradually completing configuration operations such as parameter setting and component installation. However, due to individual experience differences and interpretation deviations of network information, the configuration results often do not meet expectations, and repeated debugging becomes the norm.
[0004] Low efficiency: the environment configuration of medium complexity often takes several days or even weeks, seriously delaying the overall project progress.
[0005] No migration: after changing the hardware model or operating system version, the original configuration may be completely invalid and needs to be configured from scratch.
[0006] The above problems are more prominent in network isolated environments such as military areas: The network is physically isolated from the outside world, and compatibility problems, component conflicts and other faults in the environment cannot be diagnosed through online tools, and can only be exposed by reporting step by step, which is extremely difficult to troubleshoot.
[0007] When supplementing configuration resources, engineers need to download packages from public networks based on experience, and need to go through multiple rounds of safety verification by technical departments (detecting malicious code), security departments (reviewing information compliance), and operation and maintenance teams (verifying system adaptability). The process often takes several weeks, resulting in a several-fold increase in the deployment cycle of high-precision technologies, which seriously restricts the rapid landing and effectiveness of key technologies.
[0008] For example, Figure 1For the closest prior art "Revisiting Knowledge-Based Inference of Python Runtime Environments" (hereinafter referred to as ReadPyE), it is a practical and adaptive Python runtime environment inference method: the prior art mainly has the following three shortcomings: 1. High maintenance cost: ReadPyE is powerful, but its ecosystem knowledge graph is difficult to change, the algorithm is complex, and contains a lot of redundant knowledge, which is difficult to maintain.
[0009] 2. Lack of engineering application: Since ReadPyE cannot be updated by itself, as third-party libraries iterate, the knowledge of the knowledge graph it stores will lose its value, resulting in lower practicality and inability to achieve engineering-level applications.
[0010] 3. Lack of adaptability: In network isolated environments such as military areas, runtime environment problems can only be exposed layer by layer. When configuring the environment, engineers need to download resources from the public network and undergo rigorous security review, which seriously restricts the efficiency of deploying high-tech technologies, making it difficult for advanced technologies to be quickly implemented. SUMMARY
[0011] To solve the above problems, the present application provides a complex runtime environment rapid construction system for military and civilian fields, which significantly reduces the environment construction time and cost, breaks through the network isolation limit, has self-learning and continuous optimization capability, and realizes intelligent diagnosis of environment dependence problems.
[0012] The present application is realized by the following technical solutions: a complex runtime environment rapid construction system for military and civilian fields: The system comprises a pre-processing installation and verification module PEMS, a memory retrieval and analysis module MARS, and an automated experience extraction module AIDE; The pre-processing installation and verification module PEMS is used to receive a target Python project, obtain initial environment configuration information based on a pre-processing tool, and construct a runtime environment in an isolated environment, monitor the environment construction process and capture environment-related errors; The memory retrieval and analysis module MARS is used to receive the environment-related errors captured by the pre-processing installation and verification module PEMS, generate installation command suggestions based on a memory repository and preset rules, and return them to the pre-processing installation and verification module PEMS; The automated experience extraction module AIDE is used to extract structured minimum viable memory units MVMU from operation history after the pre-processing installation and verification module PEMS completes the environment construction process and stores them.
[0013] Further, the complex operation environment rapid construction system further comprises a memory repository, The memory repository is configured to store the minimum viable memory unit MVMU generated by the automated experience extraction module AIDE and provide retrieval for the memory retrieval and analysis module MARS.
[0014] Further, the pre-processing installation and verification module PEMS has two working stages, specifically: First, install the candidate software package in sequence based on the Conda environment. When performing the software package installation operation, set a maximum retry limit for the case of single software package installation failure; if the retry reaches the maximum retry limit and is still unsuccessful, skip the installation of the current software package and record the status; When all candidate packages are installed or skipped due to exceeding the retry limit, enter the project running verification stage. If the project runs successfully or the error is caused by environment-independent problems, the environment is considered valid; if the running error is caused by missing dependencies, the pre-processing installation and verification module PEMS will capture the error and send it to the memory retrieval and analysis module MARS for processing.
[0015] Further, the memory retrieval and analysis module MARS receives error information from the pre-processing installation and verification module PEMS; based on the pre-set error extraction rules, processes the error information to generate a core problem summary; Convert the core problem summary into a vector representation; based on the vector representation, retrieve semantically similar historical memory entries from the memory repository; combine the retrieved historical memory entries, error information, and related context information to form a new installation command suggestion based on the pre-set command generation rules; return the new installation command suggestion to the pre-processing installation and verification module PEMS for subsequent installation attempts; Whenever an environment-related error is encountered, the module calls a large language model to summarize the error to form a problem statement, and retrieves relevant experience from the memory library according to the summary, then combines the retrieved knowledge with the current context to generate a new installation suggestion.
[0016] Further, the automated experience extraction module AIDE obtains the operation history record of the current environment construction process after the pre-processing installation and verification module PEMS confirms that the target Python project runs successfully or the failure reason is non-environment error; Based on the pre-set problem-driven extraction rules, backtrack to identify the key operation path that solves the environment error from the operation history record; Based on the preset structured packaging rule, the key operation path, the corresponding original error information and the finally effective solution information are packaged into a structured minimum viable memory unit (MVMU), and the minimum viable memory unit is stored into the memory repository.
[0017] Further, the memory repository is initially empty or contains basic memory units. The automatic experience extraction module (AIDE) continuously generates new minimum viable memory units and updates them to the memory repository, realizing the self-learning ability of the system.
[0018] Further, the problem-driven extraction rule of the automatic experience extraction module (AIDE) is specifically: The extraction process is started only after the entire environment construction process is completed. Only the operation command sequence that actually solves the error in the finally successfully constructed environment is analyzed and extracted as the key operation path; the intermediate operation records that do not finally solve the problem are ignored.
[0019] Further, the complex running environment rapid construction system further comprises: A private PyPI image source service module deployed in an internal network environment; The private PyPI image source service module is built based on the bandersnatch tool and is configured to synchronize only the specified Python third-party library and all historical versions and dependency metadata thereof; The private PyPI image source service module provides an access interface by an Apache HTTP server; The pre-processing installation and verification module (PEMS) is configured to obtain software packages from the private PyPI image source service module when installing software packages in the isolated environment.
[0020] Advantages of the present application The present application proposes a complex running environment rapid construction system for military and civilian fields to solve the problems in the prior art, which has two versions: Military version: for the network isolation and high security demand of military scientific research scenes, to help scientific researchers to quickly complete environment construction in offline and limited network environment.
[0021] Civilian version: for the wide use of Python projects in various industries, to reduce the environment configuration threshold and improve the code running environment construction efficiency.
[0022] Two versions are based on an intelligent agent with self-learning ability: through the intelligent agent deployment to reduce the difficulty of individual operation, when encountering error, the memory and prompt word engineering guide the large model to provide solutions and execute, and after the reasoning ends, the valuable experience is extracted into the memory in a problem-oriented manner, solving the core pain point of ReadPyE that cannot be updated.
[0023] The military version is based on the civilian version, and collects private PyPI mirror sources and large model service and remote calling technologies to meet the use needs of the military high-security area.
[0024] The present application is based on a complex conda environment rapid construction system of python language, a complete and usable conda environment is constructed through analysis of the target project; the AI agent can independently solve error information and has self-learning ability, can analyze and process error information independently, and can backtrack valuable memories through a problem-driven memory updating mechanism; the private PyPI mirror source and local large model calling are used to realize the complete use of the project in the militarized network isolation area. BRIEF DESCRIPTION OF DRAWINGS
[0025] Figure 1 It is a knowledge graph diagram of the prior art (ReadPyE).
[0026] Figure 2 It is a private PyPI mirror source deployment architecture diagram. DETAILED DESCRIPTION
[0027] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.
[0028] The experimental methods used in the following embodiments are conventional methods unless otherwise specified. The materials, reagents, methods and instruments used are conventional materials, reagents, methods and instruments in the art unless otherwise specified, and can be obtained by commercial channels by those skilled in the art.
[0029] The present application proposes a complex operating environment rapid construction system (QuickConda) for military and civilian fields, including a pre-processing installation and verification module PEMS, a memory retrieval and analysis module MARS and an automated experience extraction module AIDE. The application is a Python environment dependency reasoning AIAgent with self-learning ability, not only relying on static knowledge, but also through simulating Python dependency installation process, abstracting the "problem-solving" path, and constructing a lightweight and migratable memory library. During the reasoning process, it retrieves related memories to assist the large model to solve problems more efficiently and accurately; after the reasoning of the complex project is completed, the installation experience in the reasoning process will be updated to the memory library. With the self-learning ability, with the user's use in VSCode and the development of artificial intelligence, QuickConda will be faster and more accurate in reasoning the project environment.
[0030] Efficient and accurate environment reasoning is achieved through the following three core modules: PEMS: pre-processing installation and verification module (Python Environment Management System); used to receive the target Python project; call the preprocessing tool to obtain the initial environment configuration information, including the Python interpreter version and the candidate package list; Based on the initial environment configuration information, the package installation operation is performed in the isolated environment in sequence; after completing the installation operation or reaching the preset skipping condition, the target Python project is run in the isolated environment; the installation operation and the project running process are monitored, and the error information related to the environment is captured; That is, this module is responsible for Python environment management; mainly divided into two different stages: pre-processing installation stage and running verification stage. The first stage, the system will install the pre-processed package sequence; in the second stage, the system will run the target project on the basis of the environment constructed in the first step to verify the integrity and correctness of the environment. Any error in either stage will be forwarded to the MARS module, which will provide more complete installation suggestions to solve the problem.
[0031] Specifically: after receiving the Python project and the correct Python project command, PEMS will call ReadPyE to preprocess the project and generate an initial Dockerfile, and then extract the Python interpreter version and the candidate package list. In the first stage, PEMS will start a Conda environment and try to install these candidate packages in sequence. In this stage, in order to avoid excessive repeated attempts or package installation failures due to reasoning errors in the preprocessing stage itself, PEMS limits the number of attempts to solve each error to five times. If the number of times is exceeded, it will continue to process the next package, but this step adjustment does not mean giving up solving the problem, on the contrary, if the problem is indeed a problem that must be solved by the project, the problem will be reported as a truly missing dependency in the running stage, so that MARS can use the improved context for analysis.
[0032] Once all the candidate packages are installed or skipped due to exceeding the number of retries (upper limit is N+12, where N is the number of candidate packages predicted during preprocessing), the system will enter the running verification phase. In the second phase, PEMS will run the Python project within the Conda environment. If the project runs successfully or the error is caused by environment-independent issues (such as logical errors in the code), the environment is considered valid. Otherwise, MARS will be called again to resolve the environment dependency error at runtime.
[0033] MARS: Memory Access and Retrieval System; for: receiving error information from the pre-processing installation and verification module PEMS; processing the error information based on pre-set error extraction rules to generate a core problem summary; converting the core problem summary into a vector representation; based on the vector representation, retrieving semantically similar historical memory entries from the memory repository; combining the retrieved historical memory entries, the error information, and the relevant context information to form a new installation command suggestion based on pre-set command generation rules; returning the new installation command suggestion to the pre-processing installation and verification module PEMS for subsequent installation attempts; Whenever an environment-related error is encountered, this module will call the large language model to summarize the error to form a problem statement, and retrieve relevant experience from the memory library according to the summary, then combine the retrieved knowledge with the current context to generate a new installation suggestion.
[0034] Specifically: when MARS receives a failed installation command and its corresponding error, it first uses prompt engineering to guide the large model to summarize and extract the core problem from the original error log. This step helps to eliminate irrelevant noise related to the code itself and avoid the influence of error outputs containing irrelevant information on the accuracy of memory retrieval. Then MARS will encode the extracted key error into an embedding vector and query the memory repository to retrieve the top 5 most similar historical memories. After the above memory retrieval preparation work is done, the retrieved relevant memories will be combined with the original installation command, error information and previous reasoning history to form a complete context to guide the large model to output. The large model will integrate the context and return a feasible installation command suggestion. The optimized command suggestion will be passed back to PEMS for the next installation attempt.
[0035] AIDE: Automated Insight & Dependency Extractor; This module parses the entire operation history after the reasoning is completed, extracts structured atomic experiences, and updates them to the memory bank to ensure the accuracy and generalizability of stored knowledge.
[0036] After the PEMS confirms that the target Python project runs successfully or the failure reason is a non-environment error, the operation history of this environment construction process is obtained; Based on the preset problem-driven extraction rules, the key operation path for solving the environment error is identified from the operation history record; Based on the preset structured encapsulation rules, the key operation path, the corresponding original error information, and the final effective solution information are encapsulated into a structured minimum viable memory unit MVMU; and the minimum viable memory unit is stored in the memory storage library; The memory storage library is used to store the minimum viable memory unit generated by the automated experience extraction module AIDE, and is used for retrieval by the memory retrieval and analysis module MARS.
[0037] The AIDE module solves the core problem of how to extract structured and generalizable memory units from the dependency reasoning process. Traditional methods usually try to record all intermediate states during the entire reasoning process, resulting in knowledge that is full of redundant and even misleading information, which significantly increases the maintenance cost and retrieval complexity. To solve this problem, a problem-driven memory construction algorithm is proposed. The core idea is to avoid recording all exploratory operations; on the contrary, only the effective error->repair path sequence in the historical operation is focused on.
[0038] Specifically, when an error occurs (such as installation failure or execution error), the AIDE module does not immediately record the failure state. Instead, it adopts a delayed extraction strategy and only backtracks the historical operation record after the entire reasoning process is completed, thereby determining the key operation path that finally solves the problem. Then, the path is passed to the large model for analysis and inductive extraction. Based on the original installation command, the result of running the code, and the error information, the large model can identify which operations really help solve the problem and encapsulate them into a minimum viable memory unit MVMU), each MVMU is a structured entry.
[0039] Embodiment: In the military area, the private PyPI mirror source is deployed, which can significantly reduce the time required for environment configuration, avoid leakage and other problems, and help the test and verification of key algorithms, model deployment and rapid transformation of technical achievements.
[0040] In the embodiment, a method for deploying a private PyPI mirror source is proposed to ensure efficient use of QuickConda in a secure intranet. The method discards the simple pip2pi to build the mirror source service, which can only save packages matching the server configuration of the mirror source service provider, and has poor migration. Therefore, the bandersnatch+Apache service technology is selected, which can customize the installation of the required third-party library list by modifying the configuration file and download all versions of the third-party library, synchronize its metadata and dependency information, and generate a directory structure that meets the PEP503 simplified index specification, thereby building a private mirror station with PyPI functions in an offline environment. The third-party library content under the mirror station can be downloaded on the public network, and the hard disk containing the third-party library data is inserted into the intranet server, and the mirror source service is started in the intranet server. In the specific implementation process, the bandersnatch tool is used as the core component, combined with the Apache HTTP server, to build a high-availability PyPI mirror system that supports access within the local area network. The data required by the private PyPI mirror source service module is imported from the external network to the internal network environment through a physical storage medium; the synchronization function based on timing tasks or incremental package updates is supported, and data updates are implemented through physical storage medium import.
[0041] The embodiment has high customizability: by modifying the configuration file, only the required specific third-party library and all its historical versions can be synchronized in the militarized network isolation area, greatly reducing the disk space occupied by the mirror source and the download time; Automation and maintainability: bandersnatch supports timing task running and incremental update function, which can automatically synchronize new library versions and dependencies every day without human intervention; Good compatibility: the mirror station structure is completely consistent with the official PyPI, and supports the standard download method of pip, which realizes the installation of dependencies from a private mirror source without changing the user's development habits; Easy to deploy and reuse: bandersnatch environment is encapsulated by Docker image, combined with persistent mounting point, one-key start and fast migration can be realized, which is convenient for deployment and reuse in different military scientific research units.
[0042] In summary, the complex operation environment quick construction system (QuickConda) proposed in the application can one-key construct the development environment required by a complex Python project, and truly realize "offline second configuration". Through the above mechanism, the construction cost of the scientific research environment in a high-isolation scene is significantly compressed, and the environment deployment is compressed from the original "several days" to "several tens of minutes", which provides strong support for front-line algorithm testing, model deployment and high-precision military scientific research project verification, and speeds up the transformation process of scientific research achievements to combat capability.
[0043] The above describes in detail the complex operation environment quick construction system proposed in the application, and describes the principle and implementation mode of the application. The above description of the embodiments is only used to help understand the method and core idea of the application. Meanwhile, for those skilled in the art, the specific implementation mode and application range will be changed according to the idea of the application. In summary, the content of the specification should not be understood as a limitation of the application.
Claims
1. A system for rapidly constructing complex operating environments for both military and civilian applications, characterized in that: The system includes a preprocessing installation and verification module PEMS, a memory retrieval and parsing module MARS, and an automated experience extraction module AIDE. The Preprocessing Installation and Verification Module (PEMS) is used to receive the target Python project, obtain the initial environment configuration information based on the preprocessing tool, build the runtime environment in the isolated environment, monitor the environment building process, and capture environment-related errors. The Memory Retrieval and Parsing Module (MARS) is used to receive the environment-related errors captured by the Preprocessing Installation and Verification Module (PEMS), generate installation command suggestions based on the memory repository and preset rules, and return them to the Preprocessing Installation and Verification Module (PEMS). The Automated Experience Extraction (AIDE) module is used to extract and store the Structured Minimum Viable Memory Unit (MVMU) from the operation history after the Preprocessing Installation and Verification (PEMS) module completes the environment building process.
2. The rapid construction system for complex operating environments according to claim 1, characterized in that: The rapid construction system for complex operating environments also includes a memory repository. The memory repository is used to store the minimum feasible memory unit (MVMU) generated by the automated experience extraction module (AIDE) and is retrieved by the memory retrieval and parsing module (MARS).
3. The rapid construction system for complex operating environments according to claim 2, characterized in that: The Preprocessing Installation and Verification Module (PEMS) has two working phases: First, candidate packages are installed sequentially based on the Conda environment. When performing package installation, a maximum retry limit is set for cases where the installation of a single package fails. If the maximum retry limit is reached and the installation still fails, the installation of the current package is skipped and its status is recorded. Once all candidate packages have been installed or skipped due to exceeding the retry limit, the project enters the project execution and verification phase. During this phase, if the project runs successfully or an error occurs due to an issue unrelated to the environment, the environment is considered valid. If an error occurs due to missing dependencies, the Preprocessing Installation and Verification module PEMS will capture the error and send it to the Memory Retrieval and Parsing module MARS for processing.
4. The rapid construction system for complex operating environments according to claim 3, characterized in that: The Memory Retrieval and Parsing Module (MARS) receives error information from the Preprocessing Installation and Verification Module (PEMS); based on preset error extraction rules, it processes the error information and generates a summary of core issues. The core problem summary is converted into a vector representation; based on the vector representation, semantically similar historical memory entries are retrieved from the memory repository; the retrieved historical memory entries, error messages, and relevant context information are combined to form new installation command suggestions based on preset command generation rules; the new installation command suggestions are returned to the preprocessing installation and verification module PEMS for subsequent installation attempts. Whenever an environment-related error is encountered, the module calls the large language model to summarize the error to form a problem statement, retrieves relevant experiences from the memory bank based on the summary, and then combines the retrieved knowledge with the current context to generate new installation recommendations.
5. The complex operating environment quick building system according to claim 4, characterized in that: The AIDE obtains the operation history record of the current environment building process after the PEMS confirms that the target Python project runs successfully or the failure reason is a non-environment error; Based on the preset problem-driven extraction rules, the key operation path for solving the environment error is identified from the operation history record; Based on the preset structured encapsulation rules, the key operation path, the corresponding original error information, and the final effective solution information are encapsulated into a structured minimum viable memory unit (MVMU); The minimum viable memory unit is stored in the memory repository.
6. The complex operating environment quick building system according to claim 5, characterized in that: The memory repository is initially empty or contains basic memory units; The AIDE continuously generates new minimum viable memory units and updates them to the memory repository, realizing the self-learning ability of the system.
7. The complex operating environment quick building system according to claim 6, characterized in that: The problem-driven extraction rules of the AIDE are as follows: The extraction process is started only after the entire environment building process is completed; Only the operation command sequence that actually solves the error in the finally successfully built environment is analyzed and extracted as the key operation path; intermediate operation records that do not finally solve the problem are ignored.
8. The complex operating environment quick building system according to claim 7, characterized in that: The complex operating environment quick building system further comprises: A private PyPI mirror source service module deployed in the internal network environment; The private PyPI mirror source service module is built based on the bandersnatch tool and is configured to synchronize only the specified Python third-party libraries and all their historical versions and dependency metadata; The private PyPI mirror source service module provides an access interface by Apache HTTP server; The PEMS is configured to obtain software packages from the private PyPI mirror source service module when installing software packages in the isolated environment.