Request processing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a request processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: in response to a login request, creating a running environment through a multi-user management center to obtain a user development environment container corresponding to the login request; when a file access request sent by an application program in the user development environment container is received, obtaining a unified target path mounted in the user development environment container through the network file system, and querying target service data in the cache according to the unified target path; if the target business data exists in the cache, reading the target business data from the cache through a unified target path; and if the target business data does not exist in the cache, obtaining the target business data from the at least one remote storage system through the unified target path, and storing the target business data to the cache. By adopting the method, the time consumption for reading the business data based on multiple data sources in a distributed environment can be reduced.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a request processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] With the rapid development of big data and artificial intelligence technologies, data processing, analysis, and machine learning modeling based on Python have become an important part of enterprises' core business capabilities. To meet the needs of multi-team collaboration, efficient resource utilization, and unified environment management, container-based distributed Python development and deployment platforms have emerged.

[0003] In traditional technical solutions, when users perform data analysis or model training in a distributed environment, business data is typically stored directly or relies on remote distributed storage systems. User code must access remote data directly over the network using dedicated client libraries and specific access protocols for each storage system.

[0004] However, traditional methods suffer from long reading times and high data latency when reading business data from multiple data sources in a distributed environment. Summary of the Invention

[0005] Therefore, it is necessary to provide a request processing method, apparatus, computer device, computer-readable storage medium, and computer program product that can reduce the time spent reading business data in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides a request processing method, the method comprising:

[0007] In response to a login request, a runtime environment is created through a multi-user management center to obtain the user development environment container corresponding to the login request; wherein, the multi-user management center is deployed on a container orchestration platform cluster;

[0008] When a file access request is received from an application in the user development environment container, a unified target path mounted in the user development environment container is obtained through the network file system, and the target business data in the cache is queried according to the unified target path; wherein, the unified target path is the path of business data and cached business data of at least one remote storage system mounted by the network file system.

[0009] If the target business data exists in the cache, the target business data is read from the cache through the unified target path in response to the file access request;

[0010] If the target business data is not present in the cache, the target business data is retrieved from at least one of the remote storage systems via the unified target path in response to the file access request; and the target business data is stored in the cache; wherein the target business data is used to train the business model.

[0011] In one embodiment, the method for constructing the unified target path includes:

[0012] Configure the connection relationship between the network file system and the at least one of the remote storage systems and the cache;

[0013] The unified target path is determined based on the aforementioned connection relationships.

[0014] In one embodiment, determining the unified target path based on the respective connection relationships includes:

[0015] Create a main shared directory;

[0016] Under the main shared directory, at least one first subdirectory corresponding to a remote storage system is created according to the connection relationship, and a second subdirectory corresponding to the cache is created to obtain the populated main shared directory; wherein, each first subdirectory corresponds to one remote storage system;

[0017] The populated main shared directory is shared to the user development environment container via the network file system protocol;

[0018] The populated main shared directory is mounted to the user development environment container to obtain the unified target path.

[0019] In one embodiment, configuring the connection relationship between the network file system and the at least one of the remote storage systems and the cache includes:

[0020] The network file system is configured to connect to at least one of the remote storage systems and to the cache using automated scripts; wherein the automated scripts include connection parameters for each of the remote storage systems and connection parameters for the cache.

[0021] In one embodiment, the method further includes:

[0022] If the current capacity of the cache reaches the target capacity threshold, delete business data whose access frequency is less than the access frequency threshold within the target time period.

[0023] In one embodiment, the file access request is determined based on the Python language; the target business data includes a first type of business data and a second type of business data; wherein, the second type of business data is data about the first type of business data.

[0024] Secondly, this application also provides a request processing apparatus, the apparatus comprising:

[0025] A module is created to respond to login requests by creating a runtime environment through a multi-user management center, thereby obtaining the user development environment container corresponding to the login request; wherein, the multi-user management center is deployed on a container orchestration platform cluster;

[0026] The receiving module is used to obtain a unified target path mounted in the user development environment container through the network file system when it receives a file access request sent by an application in the user development environment container, and query the target business data in the cache according to the unified target path; wherein, the unified target path is the path of business data and cached business data of at least one remote storage system mounted by the network file system.

[0027] The first response module is used to read the target business data from the cache through the unified target path if the target business data exists in the cache, in order to respond to the file access request;

[0028] The second response module is configured to, if the target business data is not present in the cache, retrieve the target business data from at least one of the remote storage systems via the unified target path in response to the file access request; and store the target business data in the cache; wherein the target business data is used to train the business model.

[0029] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0030] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0031] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0032] The aforementioned request processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product, firstly, in response to a login request, create a runtime environment through a multi-user management center to obtain the user development environment container corresponding to the login request; wherein, the multi-user management center is deployed on a container orchestration platform cluster; the overall deployment within the container orchestration platform cluster architecture realizes resource control over the data processing process and isolation of computing resources between users. Secondly, when a file access request is received from an application in the user development environment container, a unified target path mounted in the user development environment container is obtained through the network file system, and the target business data in the cache is queried according to the unified target path; wherein, the unified target path is the path of business data from at least one remote storage system and cached business data mounted in the network file system; cached data and data from different remote storage systems are all mounted within the unified target path, realizing unified data naming management, and also realizing direct read and write to big data clusters through distributed caching technology. Thirdly, if the target business data exists in the cache, the target business data is read from the cache through the unified target path to respond to the file access request; this avoids obtaining the target business data from multiple remote data sources and reduces the time spent reading the target business data. Finally, if the target business data is not present in the cache, it is retrieved from at least one remote storage system via a unified target path to respond to the file access request; and the target business data is stored in the cache; wherein, the target business data is used to train the business model; by retrieving the target business data from the remote storage system and storing it in the cache so that it can be retrieved directly from the cache on the next request, the time spent reading the target business data is reduced, the latency of retrieving the target business data is reduced when training the business model, and the efficiency is improved. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is an application environment diagram of a request processing method in one embodiment;

[0035] Figure 2 This is a flowchart illustrating a request processing method in one embodiment;

[0036] Figure 3 This is a flowchart illustrating the process of determining a unified target path based on various connection relationships in one embodiment.

[0037] Figure 4This is a structural block diagram of a request processing device in one embodiment;

[0038] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0040] The request processing method provided in this application embodiment can be applied to, for example, Figure 1 The application environment shown depicts a distributed system comprised of multiple physical servers, with multiple server nodes forming a Kubernetes cluster, a container orchestration platform. The data storage system can be integrated onto the Kubernetes cluster or hosted in the cloud or on other network servers. Node 1 in the Kubernetes cluster sends a login request to nodes deploying a multi-user management center (JupyterHub) and a Network File System (NFS). When a multi-user management center (JupyterHub) and a network file system node receive a file access request from an application in a user's development environment container, they obtain a unified target path mounted in the user's development environment container through the network file system. Based on this unified target path, they query the cached target business data. The unified target path is the path where the network file system mounts business data from at least one remote storage system and the cached business data. If the target business data exists in the cache, it is read from the cache through the unified target path to respond to the file access request. If the target business data does not exist in the cache, it is retrieved from at least one remote storage system through the unified target path to respond to the file access request. The target business data is then stored in the cache. This target business data is used to train the business model.

[0041] In one exemplary embodiment, such as Figure 2 As shown, a request processing method is provided, which is applied to Figure 1 Taking any node server in the Kubernetes cluster, a container orchestration platform, as an example, the explanation includes the following steps S202 to S208. Wherein:

[0042] Step S202: In response to the login request, a runtime environment is created through the multi-user management center to obtain the user development environment container corresponding to the login request.

[0043] The multi-user management center is deployed on a container orchestration platform cluster. This cluster can be Kubernetes. Node 2 is a node within the Kubernetes cluster, and it is a server that deploys the multi-user management center JupyterHub and the Network File System (NFS).

[0044] Package the multi-user management platform center JupyterHub into a container image and then deploy it on at least one node of the Kubernetes cluster: JupyterHub can be used to create a multi-user JupyterLab, which can generate, manage and proxy multiple instances of a single-user JupyterLab server. The JupyterHub installation package is made into an image and deployed on the K8s cluster to achieve containerized deployment.

[0045] JupyterLab allows for data analysis and modeling, interactive programming, and unified management and authentication through JupyterHub. For deployment, the JupyterLab installation package is created as an image and deployed on a Kubernetes cluster, achieving containerized deployment.

[0046] The main features of JupyterLab include the following:

[0047] Python's interactive mode allows you to directly input code, execute it, and get the results immediately. Therefore, Python's interactive mode is mainly used for debugging Python code.

[0048] Code can be enabled in any text file running within the JupyterLab kernel.

[0049] The modular interface allows you to open several notebooks or files in the same window at the same time, all displayed in tabs, making it more like an IDE.

[0050] Multiple views of the same document allow users to edit documents and view results in real time.

[0051] It supports multiple data formats, allowing users to view and process various data formats, and also provides rich visualization output.

[0052] Optionally, a credit card fraud detection model needs to be trained on node 1. Node 1 receives login requests from users, for example, when user "Zhang San" logs into the multi-user management center JupyterHub through a browser on the terminal, a login request is generated. The login request is sent to the server, and the server responds to the login request by creating a runtime environment through the multi-user management center JupyterHub, starting an independent user development environment Jupyterlab container instance for Zhang San's terminal, i.e., the user development environment Jupyterlab container. Node 1 has its own independent file system, network, and process space.

[0053] Optionally, when user Li Si logs in, node 2 will also start another completely independent container instance for "Li Si". This instance originates from the same image as "Zhang San's" container, but they do not interfere with each other. In traditional methods, multiple users share a single server. Without isolation, a task run by one user may exhaust all CPU and memory, causing other users' services to crash. Kubernetes can precisely allocate and limit CPU and memory resources for each container. No matter how resource-intensive user "Zhang San's" model training is, it will not affect user "Li Si's" container.

[0054] Step S204: When a file access request is received from an application in the user development environment container, the unified target path mounted in the user development environment container is obtained through the network file system, and the target business data in the cache is queried according to the unified target path.

[0055] The unified target path is the path on which business data and cached business data from at least one remote storage system are mounted in the network file system. The unified target path serves as the unified entry point for accessing the target business data. The cache refers to the local cache of a server, such as node 2.

[0056] At this point, a JupyterLab container instance running in the user development environment is located on node 1. Applications running within this container, such as business model training programs, send file access requests to node 2. Node 2 receives the file access request, obtains the unified target path mounted within the JupyterLab container via the Network File System (NFS), and then queries the cached target business data based on that unified target path.

[0057] In multi-source data scenarios, such as the distributed file system HDFS and other remote storage, each has its own complex access addresses. By mounting these completely different remote storage systems to a single unified target path, data on HDFS and other storage systems becomes local file paths. Therefore, a unified target path solves the problems of dispersed data sources and complex management.

[0058] Step S206: If the target business data exists in the cache, the target business data is read from the cache through the unified target path to respond to the file access request.

[0059] Optionally, if the target business data exists in the cache, Node 2 reads the target business data from its local cache via a unified target path to respond to the file access request, without needing to access the remote HDFS. The target business data is then returned to the user's development environment JupyterLab container. For example, if the file access request was initiated by Node 1 where Zhang San resides, the target business data is returned to Node 1 where Zhang San resides.

[0060] Step S208: If the target business data does not exist in the cache, the target business data is obtained from at least one remote storage system through a unified target path to respond to the file access request; and the target business data is stored in the cache.

[0061] The target business data is used to train the business model.

[0062] Optionally, if the target business data is not present in the cache, Node 2 retrieves the target business data from at least one remote storage system through a unified target path in response to the file access request; and stores the target business data in the cache, leaving a copy, which greatly reduces data access latency.

[0063] Node 2 obtains target business data from at least one remote storage system through a unified target path. That is, it obtains data from different data sources through a unified entry point, avoiding the need to issue different data retrieval instructions for different data sources, and thus avoiding the complexity of programming the retrieval instructions.

[0064] In the above request processing method, firstly, in response to a login request, a runtime environment is created through a multi-user management center to obtain the user development environment container corresponding to the login request. The multi-user management center is deployed on a container orchestration platform cluster. The overall deployment within the container orchestration platform cluster architecture enables resource control over the data processing process and isolation of computing resources between users. Secondly, when a file access request is received from an application in the user development environment container, a unified target path mounted in the user development environment container is obtained through the network file system. The target business data in the cache is then queried based on this unified target path. The unified target path is the path on the network file system that mounts business data from at least one remote storage system and cached business data. Data from the cache and different remote storage systems is mounted within the unified target path, achieving unified data naming management. Distributed caching technology also enables direct read / write access to the big data cluster. Thirdly, if the target business data exists in the cache, it is read from the cache through the unified target path to respond to the file access request. This avoids obtaining target business data from multiple remote data sources, reducing the time spent reading target business data. Finally, if the target business data is not present in the cache, it is retrieved from at least one remote storage system via a unified target path to respond to the file access request; and the target business data is stored in the cache; wherein, the target business data is used to train the business model; by retrieving the target business data from the remote storage system and storing it in the cache so that it can be retrieved directly from the cache on the next request, the time spent reading the target business data is reduced, the latency of retrieving the target business data is reduced when training the business model, and the efficiency is improved.

[0065] In one exemplary embodiment, the construction of a unified target path includes: configuring the connection relationships between the network file system and at least one remote storage system and a cache; and determining the unified target path based on each connection relationship.

[0066] Remote storage systems can include distributed file systems and object storage systems.

[0067] Optionally, the server configures the connection relationships between the network file system and at least one remote storage system, as well as between the network file system and the cache; for example, configuring the connection relationship between NFS and HDFS, such as configuring the path in HDFS, the path exposed in NFS, HDFS port, etc.; the server also configures NFS and an object storage system. The server further configures NFS and a cached object storage system. Based on the configured connection relationships, the server determines a unified target path.

[0068] In this embodiment, by establishing a unified target path, users can access data in distributed storage as if it were a local file, while all complex operations such as caching, acceleration, and protocol conversion are automatically completed in the background.

[0069] In one exemplary embodiment, such as Figure 3 As shown, a unified target path is determined based on the various connection relationships, including steps S302 to S308. Wherein:

[0070] Step S302: Create the main shared directory.

[0071] Step S304: In the main shared directory, create at least one first subdirectory corresponding to the remote storage system and a second subdirectory corresponding to the cache according to the connection relationship, so as to obtain the populated main shared directory.

[0072] Each first subdirectory corresponds to a remote storage system.

[0073] Optionally, the server creates an empty main shared directory, such as `mkdir / export / bank_data_center`. Within this main shared target, at least one first subdirectory and one second subdirectory are created based on the connection relationships. For example:

[0074] create a directory named / export / bank_data_center / hdfs_window / / first subdirectory

[0075] create a directory named / export / bank_data_center / Cache_window / / second subdirectory

[0076] In the main shared directory, such as mkdir / export / bank_data_center, the first subdirectory: mkdir / export / bank_data_center / hdfs_window and the second subdirectory: mkdir / export / bank_data_center / Cache_window are populated.

[0077] Step S306: Share the populated main shared directory to the user's development environment container via the network file system protocol.

[0078] Optionally, the server can share the populated main shared directory with the user's development environment container, such as Zhang San's JupyterLab container, via a network file system protocol.

[0079] Step S308: Mount the populated main shared directory into the user development environment container to obtain a unified target path.

[0080] Optionally, when Zhang San's JupyterLab container starts, the server automatically mounts the populated main shared directory into Zhang San's JupyterLab container, such as `mount -t nfs nfs-server-ip: / export / bank_data_center / mnt / unified_data`. Afterwards, a new directory ` / mnt / unified_data` appears in Zhang San's container, containing two subdirectories: `hdfs_window` and `Cache_window`. This avoids the presence of multiple file access requests for accessing multiple data sources within Zhang San's container.

[0081] In this embodiment, by unifying the target path, the workflow for accessing target business data from multiple data sources is greatly simplified.

[0082] In one exemplary embodiment, configuring the connection relationship between the network file system and at least one remote storage system and the cache includes: configuring the connection relationship between the network file system and at least one remote storage system and the connection relationship between the network file system and the cache through an automated script.

[0083] The automation scripts include connection parameters for each remote storage system and connection parameters for the cache. The connection parameters for the remote storage system include information such as the path configured in the remote storage system, the path exposed in NFS, and the remote storage system port; the connection parameters for the cache include information such as the path in the cache, the path exposed in NFS, and the cache port.

[0084] Optionally, the server configures the connection relationship between the network file system and at least one remote storage system using connection parameters of the remote storage system in the automation script; and configures the connection relationship between the network file system and the cache using connection parameters of the cache in the automation script. The remote storage system includes at least one of the following: HDFS distributed file system and object storage system.

[0085] In this embodiment, by executing the aforementioned automated configuration script, the unified data access layer can be deployed and initialized with a single click. This method encapsulates the complex storage mounting and cache configuration process, which originally required manual, step-by-step execution, into a repeatable and version-manageable automated operation. This greatly reduces the complexity of system operation and maintenance, improves the consistency and reliability of deployment, and fundamentally solves the problem of high data access latency in traditional solutions.

[0086] In one exemplary embodiment, the request processing method further includes: deleting business data whose access frequency is less than the access frequency threshold within the target time period, provided that the current capacity of the cache has reached the target capacity threshold.

[0087] Among them, the target capacity threshold is less than the total capacity of the cache.

[0088] Optionally, the server maintains the cache according to the caching strategy, which includes the cache space size (i.e., cache capacity), the current cache capacity, and the cache eviction algorithm. The cache eviction algorithm evicts business data whose access frequency is less than a threshold within a target time period.

[0089] Optionally, if the current capacity of the cache reaches the target capacity threshold, delete business data with an access frequency lower than the access frequency threshold within the target time period, such as within a week, that is, delete cold data and retain hot data in the cache.

[0090] In this embodiment, by evicting business data from the cache, the speed and efficiency of obtaining the target business data can be guaranteed, and the latency of obtaining the target business data can be reduced.

[0091] In one exemplary embodiment, the file access request is determined based on the Python language; the target business data includes a first type of business data and a second type of business data; wherein, the second type of business data is data about the first type of business data.

[0092] The file access requests are determined using the Python language, enabling direct read and write operations on the big data cluster. The target business data includes two types: Type 1 business data and Type 2 business data. Type 2 business data refers to data about Type 1 business data, i.e., metadata. Metadata refers to descriptive information used to locate, identify, and manage files; it includes basic attributes, directory information, and the distribution of metadata.

[0093] In addition to caching the first type of business data locally, the server can also cache metadata, i.e., the second type of business data, locally. Previously, retrieving metadata via the network was slow; using NFS allows retrieving metadata directly from the local cluster, significantly reducing latency. Furthermore, the metadata requirements for model training are very high.

[0094] In this embodiment, by combining Python with JupyterLab and NFS, the time spent reading business data based on multiple data sources in a distributed environment is reduced; at the same time, a second type of business data, namely metadata, can be cached, which will greatly shorten the latency.

[0095] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0096] Based on the same inventive concept, this application also provides a request processing apparatus for implementing the request processing method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more request processing apparatus embodiments provided below can be found in the limitations of the request processing method described above, and will not be repeated here.

[0097] In one exemplary embodiment, such as Figure 4 As shown, a request processing apparatus is provided, including: a creation module 401, a receiving module 402, a first response module 403, and a second response module 404, wherein:

[0098] Module 401 is created to respond to login requests by creating a runtime environment through the multi-user management center to obtain the user development environment container corresponding to the login request; the multi-user management center is deployed on the container orchestration platform cluster.

[0099] The receiving module 402 is used to obtain the unified target path mounted in the user development environment container through the network file system when it receives a file access request sent by an application in the user development environment container, and query the target business data in the cache according to the unified target path; wherein, the unified target path is the path of business data and cached business data mounted by at least one remote storage system in the network file system.

[0100] The first response module 403 is used to read the target business data from the cache through a unified target path if the target business data exists in the cache, in order to respond to the file access request.

[0101] The second response module 404 is used to retrieve the target business data from at least one remote storage system through a unified target path to respond to the file access request if the target business data does not exist in the cache; and to store the target business data in the cache; wherein the target business data is used to train the business model.

[0102] In one exemplary embodiment, the request processing apparatus further includes a unified target path construction module for configuring the connection relationships between the network file system and at least one remote storage system and a cache; and determining a unified target path based on each connection relationship.

[0103] In an exemplary embodiment, the unified target path construction module is further configured to create a main shared directory; under the main shared directory, create at least one first subdirectory corresponding to a remote storage system and a second subdirectory corresponding to a cache according to the connection relationship, to obtain the populated main shared directory; wherein, each first subdirectory corresponds to a remote storage system; share the populated main shared directory to the user development environment container through the network file system protocol; and mount the populated main shared directory to the user development environment container to obtain the unified target path.

[0104] In an exemplary embodiment, the unified target path construction module is further configured to configure the connection relationship between the network file system and at least one remote storage system, as well as the connection relationship between the network file system and the cache, through an automated script; wherein the automated script includes connection parameters for each remote storage system and connection parameters for the cache.

[0105] In one exemplary embodiment, a cache capacity detection module is also included, which is used to delete business data whose access frequency is less than the access frequency threshold within the target time period when the current capacity of the cache reaches the target capacity threshold.

[0106] In one exemplary embodiment, the file access request is determined based on the Python language; the target business data includes a first type of business data and a second type of business data; wherein, the second type of business data is data about the first type of business data.

[0107] Each module in the aforementioned request processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can invoke and execute the operations corresponding to each module.

[0108] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores requested data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a request processing method.

[0109] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0110] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0111] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0112] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0113] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0114] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0115] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A request processing method, characterized in that, The method includes: In response to a login request, a runtime environment is created through a multi-user management center to obtain the user development environment container corresponding to the login request; wherein, the multi-user management center is deployed on a container orchestration platform cluster; When a file access request is received from an application in the user development environment container, a unified target path mounted in the user development environment container is obtained through the network file system, and the target business data in the cache is queried according to the unified target path; wherein, the unified target path is the path of business data and cached business data of at least one remote storage system mounted by the network file system. If the target business data exists in the cache, the target business data is read from the cache through the unified target path in response to the file access request; If the target business data is not present in the cache, the target business data is retrieved from at least one of the remote storage systems via the unified target path in response to the file access request; and the target business data is stored in the cache; wherein the target business data is used to train the business model.

2. The method according to claim 1, characterized in that, The method for constructing the unified target path includes: Configure the connection relationship between the network file system and the at least one of the remote storage systems and the cache; The unified target path is determined based on the aforementioned connection relationships.

3. The method according to claim 2, characterized in that, Determining the unified target path based on the various connection relationships includes: Create a main shared directory; Under the main shared directory, at least one first subdirectory corresponding to a remote storage system is created according to the connection relationship, and a second subdirectory corresponding to the cache is created to obtain the populated main shared directory; wherein, each first subdirectory corresponds to one remote storage system; The populated main shared directory is shared to the user development environment container via the network file system protocol; The populated main shared directory is mounted to the user development environment container to obtain the unified target path.

4. The method according to claim 2, characterized in that, The configuration of the connection relationship between the network file system and the at least one of the remote storage systems and the cache includes: The network file system is configured to connect to at least one of the remote storage systems and to the cache using automated scripts; wherein the automated scripts include connection parameters for each of the remote storage systems and connection parameters for the cache.

5. The method according to claim 1, characterized in that, The method further includes: If the current capacity of the cache reaches the target capacity threshold, delete business data whose access frequency is less than the access frequency threshold within the target time period.

6. The method according to claim 1, characterized in that, The file access request is determined based on the Python language; the target business data includes a first type of business data and a second type of business data; wherein, the second type of business data is data about the first type of business data.

7. A request processing apparatus, characterized in that, The device includes: A module is created to respond to login requests by creating a runtime environment through a multi-user management center, thereby obtaining the user development environment container corresponding to the login request; wherein, the multi-user management center is deployed on a container orchestration platform cluster; The receiving module is used to obtain a unified target path mounted in the user development environment container through the network file system when it receives a file access request sent by an application in the user development environment container, and query the target business data in the cache according to the unified target path; wherein, the unified target path is the path of business data and cached business data of at least one remote storage system mounted by the network file system. The first response module is used to read the target business data from the cache through the unified target path if the target business data exists in the cache, in order to respond to the file access request; The second response module is configured to, if the target business data is not present in the cache, retrieve the target business data from at least one of the remote storage systems via the unified target path in response to the file access request; and store the target business data in the cache; wherein the target business data is used to train the business model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images