Code slice storage method, electronic device, storage medium and program product
By acquiring information about the target code shards and the server hardware configuration, storage priorities are determined and storage strategies are generated, solving the problem of insufficient flexibility in code shard storage methods and achieving efficient storage operations and resource utilization.
Patent Information
- Application Number
- CN202511462571.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-14
- Publication Date
- 2026-01-27
- Estimated Expiration
- 2045-10-14
AI Technical Summary
The current code sharding storage method has low flexibility, which leads to a shortened hardware replacement cycle for storage devices with limited erase and write life such as level 4 cell flash memory when high-frequency data writing occurs, increasing maintenance costs and failing to meet the needs of low-latency scenarios.
By acquiring the identification information, structural information, and reference relationship information of the target code fragment, as well as the server's hardware configuration information, the target storage priority is determined, and a storage strategy suitable for the target code fragment is generated to achieve flexible storage of the target code fragment.
It improves the flexibility of storage operations, avoids response latency caused by storing frequently used code shards on devices with slow read/write speeds, avoids resource waste caused by storing infrequently used code shards on devices with fast read/write speeds, and optimizes server load.
Smart Images

Figure CN120950004B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of storage technology, and in particular to storage methods, electronic devices, storage media and program products for code sharding. Background Technology
[0002] With the rapid development of web applications, front-end engineering, server storage architecture, and edge computing are showing a trend of deep integration. In the process of storing web application code, upon receiving a front-end request, the storage path of the code shards can be parsed. Based on the parsed storage path and the pre-built mapping relationship between storage paths and storage types, the corresponding storage type is determined, and the code shards are stored in the storage device of that storage type.
[0003] This approach, which stores code fragments through fixed mapping relationships, offers limited flexibility. Furthermore, it fails to enable direct interaction between the front-end code and the storage device; data transmission requires multiple layers of scheduling by the operating system, resulting in additional performance overhead. This approach cannot meet the demands of low-latency scenarios. For storage devices with limited write / erase cycles, such as Quad-Level Cell (QLC) flash memory, repeated high-frequency data writing accelerates wear and tear, leading to shorter hardware replacement cycles and increased maintenance costs. Summary of the Invention
[0004] This application provides a method, apparatus, electronic device, storage medium, and program product for storing code fragments to address the problem of low flexibility in related technologies.
[0005] This application provides a method for storing code fragments, including:
[0006] Obtain the identification information and code structure information of the target code fragment, the reference relationship information between other code fragments in the target code file and the target code fragment, and the hardware configuration information of the server;
[0007] Based on the code structure information and reference relationship information, determine the target storage priority of the target code fragment;
[0008] Based on the target code shard's identification information, hardware configuration information, and target storage priority, a target storage strategy for the target code shard is generated.
[0009] A target storage strategy is adopted to perform storage operations on target code fragments.
[0010] This application also provides a storage device for code fragmentation, including:
[0011] The acquisition module is used to acquire the identification information and code structure information of the target code fragment, the reference relationship information between other code fragments in the target code file and the target code fragment, and the hardware configuration information of the server;
[0012] The determination module is used to determine the target storage priority of the target code fragment based on code structure information and reference relationship information;
[0013] The generation module is used to generate the target storage strategy for the target code shards based on the identification information, hardware configuration information, and target storage priority of the target code shards.
[0014] The storage module is used to perform storage operations on target code fragments using the target storage strategy.
[0015] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing any of the above-described code segmentation storage methods when executing the computer program.
[0016] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of any of the above-described code fragmentation storage methods.
[0017] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described code segmentation storage methods.
[0018] This application, after obtaining the identification information, code structure information, reference relationship information, and server hardware configuration information of the target code fragment, allows for the analysis of the code structure and reference relationship information to determine the target storage priority of the target code fragment. Furthermore, based on the target storage priority, the identification information of the target code fragment, and the hardware configuration information, a target storage strategy suitable for the characteristics of the target code fragment is generated. Finally, the target storage strategy can be used to store the target code fragment. This approach, storing the target code fragment according to its actual structure and the server's real-time hardware conditions, ensures that storage operations meet both the usage requirements of the target code fragment and the server's load, offering high flexibility. Moreover, it avoids the response latency problem caused by storing frequently used code fragments in storage devices with slow read / write speeds, and also avoids the resource waste caused by storing infrequently used code fragments in storage devices with fast read / write speeds. Attached Figure Description
[0019] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 A schematic diagram of a server architecture provided for an embodiment of this application;
[0021] Figure 2 A flowchart illustrating a code fragment storage method provided in an embodiment of this application;
[0022] Figure 3 A schematic diagram of a server hierarchy provided for an embodiment of this application;
[0023] Figure 4 A schematic flowchart of a code fragmentation storage device provided in an embodiment of this application;
[0024] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.
[0026] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.
[0027] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0028] The code fragment storage method provided in this application can be implemented by a server, such as... Figure 1As shown, at the hardware level, a server may include a Graphics Processing Unit (GPU), a Baseboard Management Controller (BMC), a Field-Programmable Gate Array (FPGA), a storage controller, storage devices, and multiple memory access nodes. Each memory access node may include a Central Processing Unit (CPU), a memory channel, and memory, with the CPU and memory connected via the memory channel.
[0029] The embodiments of this application provide a method for storing code fragments, which can be executed by the aforementioned server, such as... Figure 2 As shown, the specific processing steps of the code sharding storage method may include:
[0030] Step S201: Obtain the identification information and code structure information of the target code fragment, the reference relationship information between other code fragments in the target code file and the target code fragment, and the hardware configuration information of the server.
[0031] In this context, a target code chunk is any one of multiple code chunks included in the target code file. Each code chunk implements a specific code function. The identifier for a target code chunk can be its code name. The target code file can be JavaScript code.
[0032] Specifically, after completing the writing of the target code file, the technical staff can upload it to the server's build tool, triggering the build plugin (e.g., the "Webpack / Vite plugin") within the build tool to execute the build process. During the code transfer phase of the build process (e.g., after executing babel-loader), the build tool can invoke an Abstract Syntax Tree (AST) topology analysis engine to perform code parsing. This AST topology analysis engine can call an AST parsing library (e.g., "Acorn") to perform syntax parsing on each code shard, obtaining an AST structure corresponding to each code shard. The AST structure can include node type, position, and relationships (such as the relationship between the current node and other nodes). Taking a target code shard as an example, the server can traverse the AST structure corresponding to the target code shard to obtain the code structure information of the target code shard.
[0033] In addition, the server can periodically (for example, every 500ms) monitor its hardware configuration information. This hardware configuration information can include the hardware configuration information of each memory access node, each storage controller, and each storage device within the server.
[0034] The hardware configuration information for memory access nodes can include the number of memory access nodes, total memory capacity, memory usage, remaining memory capacity, number of cores, and core load (which can be the average utilization over one minute). Memory access nodes can be Non-Uniform Memory Access (NUMA) nodes. Core load can be obtained using the "mpstat" command, while the number of memory access nodes, total memory capacity, memory usage, remaining memory capacity, and number of cores can be obtained using the "numactl --hardware" command.
[0035] The hardware configuration information of the storage controller may include Input / Output Operations Per Second (IOPS), queue depth, average response time, and storage type. The storage type of the storage controller may be Non-Volatile Memory Express (NVME).
[0036] The hardware configuration information of the storage controller can be obtained using the command "nvme list-ctrl / dev / nvme0".
[0037] Hardware configuration information for storage devices can include storage type, wear level (WL), remaining life percentage, and total bytes written. Storage type can be a solid-state drive (SSD) or a hard disk drive (HDD). The SSD can specifically use Quad-Level Cell (QLC) flash memory technology. This hardware configuration information can be obtained by the server executing the command "smartctl -A / dev / nvme0". The wear level can be obtained using the command "nvme admin -passthru / dev / nvme0 --opcode=0x80".
[0038] Step S202: Determine the target storage priority of the target code fragment based on the code structure information and reference relationship information.
[0039] Specifically, since different code shards are used at different frequencies, and the frequency of use is related to the code structure information and reference relationship information of the code shards, the target storage priority of the target code shard can be determined by analyzing the code structure information and reference relationship information.
[0040] Step S203: Generate the target storage strategy for the target code shard based on the target code shard's identification information, hardware configuration information, and target storage priority.
[0041] Specifically, since different storage priorities apply to different hardware, and the hardware configuration information includes the configuration information of various server hardware components, the hardware configuration information applicable to the target code shard can be selected from the target storage priority and the hardware configuration information. Furthermore, to ensure accurate execution of storage operations using the corresponding storage strategy in the future, a target storage strategy can be generated based on the target code shard's identification information and the hardware configuration information applicable to the target code shard.
[0042] Step S204: Using the target storage strategy, perform storage operations on the target code fragments.
[0043] Specifically, the server can select specified hardware within the server and use the specified hardware to store the target code fragments in the server, based on the indication information of the target storage policy.
[0044] The code shard storage method of this application, after obtaining the identification information, code structure information, reference relationship information, and server hardware configuration information of the target code shard, can first analyze the code structure information and reference relationship information to determine the target storage priority of the target code shard. Further, based on the target storage priority, the identification information of the target code shard, and the hardware configuration information, a target storage strategy suitable for the characteristics of the target code shard is generated. Finally, the target storage strategy can be used to store the target code shard. In this way, by storing the target code shard according to its actual structure and the real-time hardware conditions of the server, the storage operation satisfies both the usage requirements of the target code shard and takes into account the server's load (i.e., it realizes the interaction between the front-end code and the storage device), resulting in high flexibility. Furthermore, it avoids the response latency problem caused by storing frequently used code shards in storage devices with slow read / write speeds, and also avoids the resource waste problem caused by storing infrequently used code shards in storage devices with fast read / write speeds.
[0045] Furthermore, related technologies primarily allocate front-end resources to the nearest edge node based on the user's physical location to reduce network latency. However, this approach ignores the differences in hardware configuration among edge nodes within the same geographical area. For example, some nodes may be equipped with high-performance NVMe storage devices, while others may only use slower Serial Advanced Technology Attachment (SATA) interface storage. Because these hardware performance differences are not considered, resources may be allocated to nearby but lower-performance nodes, resulting in underutilization of high-performance hardware and overall system inefficiency. This solution addresses these issues.
[0046] In some optional implementations, in step S202 above, the server may use the following specific steps to determine the target storage priority of the target code fragment based on code structure information and reference relationship information:
[0047] Step 1: Determine the storage weight value of the target code fragment based on the code structure information and reference relationship information.
[0048] Step 2: Based on the stored weight value, determine the target weight value range to which the stored weight value belongs from among multiple preset weight value ranges.
[0049] Step 3: Determine the storage priority corresponding to the target weight value range as the target storage priority.
[0050] Specifically, in large applications (e.g., edge computing applications adapted to in-vehicle terminals, IoT-related applications, cloud gaming, etc.), the applications consist of a large number of code shards with significant structural differences, meaning their usage frequencies vary considerably. Setting a corresponding storage strategy for each code shard would be costly. Therefore, the server can pre-set multiple different weight value ranges and determine the storage weight value of the target code shard based on code structure and reference relationships. After calculating the storage weight value, the server can determine the target weight value range within the pre-set ranges. Each pre-set weight value range can be assigned a corresponding storage priority; therefore, the storage priority corresponding to the target weight value can be defined as the target storage priority, allowing for the subsequent determination of the storage strategy corresponding to that priority.
[0051] For example, the preset storage priorities are high storage priority, medium storage priority, and low storage priority in descending order. The weight value range corresponding to high priority is "greater than 0.7", the weight value range corresponding to medium priority is "0.3-0.7", and the weight value range corresponding to low priority is "less than 0.3". When the determined storage weight value is 0.5, the medium storage priority can be determined as the target storage priority.
[0052] In this way, by combining code structure information (such as function call depth, module complexity, and lines of code) and reference relationship information (such as call frequency, reference level, and cross-module references), the server can more comprehensively evaluate the importance and usage frequency of code shards. This makes the calculation of storage weight values more accurate, ensuring that high-value, high-frequency code shards receive higher storage priority, which is beneficial for improving the overall access efficiency and response speed of the application. Furthermore, with technological improvements and the introduction of new features, the structure of code shards may change. Therefore, using the above method, when the structure of code shards changes, storage priorities can be redefined for storage operations, providing greater flexibility.
[0053] In some alternative implementations, code structure information may include the number of asynchronous code nodes, the number of valid lines of code, and performance type flags.
[0054] Here, an asynchronous code node can be a code node that causes the current execution flow to stop during code execution, and only resumes the current execution flow after receiving the result from the target code node. The target code node is a code node in a code slice of the target code file. An asynchronous code node can be a node waiting for asynchronous operations or a function call node. For example, a node waiting for asynchronous operations can be an "Await Expression," and a function call node can be a "Call Expression related to Fetch or axios." In step S201, the server can determine the number of asynchronous code nodes by traversing each code node in the abstract syntax tree structure of the target code slice.
[0055] Valid code refers to the actual code in a code slice that implements the relevant functionality, excluding blank lines and comment lines. In other words, the number of valid code lines is the number of lines of code in the code slice that exclude blank lines and comment lines. In step S201, the server can use the source location information of the target code slice to calculate the number of valid code lines in the target code slice.
[0056] Performance type flags can be used to indicate whether a code shard has high-performance computing requirements. In step S201, the server can determine whether the import declaration node of the target code shard contains a filename with a preset extension, or whether the target code shard calls a function module of a preset type. When it is determined that the import declaration node of the target code shard contains a filename with a preset extension, or that the target code shard calls a function module of a preset type, the performance type flag can be set to a first preset flag, which indicates that the target code shard has high-performance computing requirements. When it is determined that the import declaration node of the target code shard does not contain a filename with a preset extension, and the target code shard does not call a function module of a preset type, the performance type flag can be set to a second preset flag, which indicates that the target code shard does not have high-performance computing requirements. For example, the preset extension is ".wasm", and the preset type function module is "WebAssembly.instantiate()", where "wasm" is short for WebAssembly, which is typically used to perform computationally intensive tasks (image processing, artificial intelligence inference, etc.). The performance type flag can be "wasmUsage". The first and second preset flags can be Boolean values. The first preset flag can be "True" and the second preset flag can be "False".
[0057] The reference relationship information (also known as dependency relationship information) may include a first number of references to other code segments by the target code segment and a second number of references to other code segments by the target code segment. In step S201, the server can obtain the dependency graph of the target code segment during the compilation process and identify the reference relationship information of the target code segment from it.
[0058] Accordingly, in step one of step S202 above, the server can specifically determine the storage weight value of the target code fragment using the following steps:
[0059] Step 1: Determine the asynchronous operation density of the target code slice based on the number of asynchronous code nodes and the number of effective lines of code.
[0060] Here, asynchronous operation density means that an asynchronous operation is executed once for every n lines of code.
[0061] Step 2: Determine the importance index value of the target code fragment in the target code file based on the first quantity and the second quantity.
[0062] Step 3: Determine the storage weight value based on the performance type label, asynchronous operation density, and importance index value.
[0063] Specifically, the server can determine the asynchronous operation density of the target code fragment as the ratio of the number of asynchronous code nodes to the number of effective lines of code. During the calculation, only two decimal places can be retained; for example, an asynchronous operation density of 0.25 means that every four lines of code contain one asynchronous operation.
[0064] A code fragment with a high dependency count (secondary quantity) indicates that it relies on many other code fragments to complete its functionality, implying weak independence. Conversely, a code fragment with a high dependency count (primary quantity) indicates strong independence, suggesting it is a fundamental and important part of the application. Considering only one parameter, dependency count or dependency frequency, may lead to inaccurate assessment of importance metrics. Therefore, the server can combine both primary and secondary quantities to determine the importance metric (also known as dependency centrality) of the target code fragment within the target code file.
[0065] In addition, the server can obtain the value corresponding to the performance type flag based on the performance type flag. For example, when the performance type flag is determined to be the first preset flag, the first preset value (which can be 1) is determined to be the value corresponding to the performance type flag. Or, when the performance type flag is determined to be the second preset flag type, the first preset value (which can be 0) is determined to be the value corresponding to the performance type flag.
[0066] Finally, the server can obtain the storage weight value by weighting the values corresponding to the performance type tag, asynchronous operation density, reference centrality, etc.
[0067] For example, step 2 can be expressed as follows:
[0068] Importance index value = First quantity / (First quantity + Second quantity + 1) (1)
[0069] Step 3 can be expressed as follows:
[0070] Storage weight value = k1 × asynchronous operation density + k2 × reference centrality + k3 × value corresponding to performance type tag (2)
[0071] Where k1 can be 0.6, k2 can be 0.3, and k3 can be 0.1.
[0072] In this way, the asynchronous operation density can reflect the load and latency sensitivity of the target code fragment during runtime, the importance index value can reflect the structural position of the target code fragment in the target code file, and the performance type flag can reflect whether the function performed by the target code fragment has high performance requirements. Therefore, by combining the above three types of indicators to calculate the storage weight value, the evaluation of the target code fragment is more comprehensive and accurate. Consequently, a more suitable storage strategy can be generated to store the target code fragment, which is more in line with the actual needs of the target code fragment.
[0073] In some optional implementations, the reference relationship information may include the number of shortest paths through the target code segment and the number of shortest paths between all code segments. Accordingly, in step S201, the server can analyze the dependency graph described above, determine the shortest path between any two code segments, and count the total number of these shortest paths. Furthermore, it can determine the number of target code segments traversed by these shortest paths.
[0074] Accordingly, the aforementioned importance index values can be calculated in the following manner:
[0075] The ratio of the number of shortest paths in a target code fragment to the number of shortest paths between all target code fragments is defined as the reference centrality.
[0076] The importance index value can range from 0 to 1.
[0077] For example, if a target code file contains 5 code segments A, B, C, D, and E, and there are a total of 20 possible paths, assuming that 10 of them are the shortest paths, and B appears in 8 of the shortest paths, then the importance index value can be calculated to be 0.8, indicating that the target code segment is a necessary code segment called when 80% of the code segments in the target code file are executed.
[0078] In some optional implementations, in step S203 above, the server may specifically use the following steps to generate a target storage strategy for the target code fragment based on the target code fragment's identifier information, hardware configuration information, and target storage priority:
[0079] Step 1: Obtain hardware filtering instruction information corresponding to the target storage priority.
[0080] Step 2: Generate the target storage strategy based on the target code fragment identification information, hardware filtering instruction information, and hardware configuration information.
[0081] Specifically, the server can pre-store hardware filtering instructions corresponding to multiple storage priorities. After determining the target storage priority, the server can select the hardware filtering instruction corresponding to the target storage priority from the multiple storage priority information. Then, based on the selected hardware filtering instruction and hardware configuration information, the server can filter out hardware that meets the requirements of the target code shard, and generate a target storage policy based on the configuration information of the filtered hardware and the identification information of the target code shard.
[0082] In some optional implementations, the hardware filtering indication information corresponding to different storage priorities is different, and the specific process of generating the target storage strategy also differs. Accordingly, step two in step S203 above will be described in two categories below.
[0083] Scenario 1: When the target storage priority is the lowest among multiple preset storage priorities, the hardware filtering indication information includes the first storage type, and the hardware configuration information includes the storage types of multiple storage devices. For example, the first storage type in the hardware filtering indication information could be an HDD in a Redundant Array of Independent Disks Level 5 (RAID5) array.
[0084] The server can determine the storage device corresponding to the first storage type from multiple storage devices based on the first storage type and the storage type of at least one storage device. It then obtains the storage path of the storage device corresponding to the first storage type. Based on the storage path of the storage device corresponding to the first storage type and the identifier information of the target code fragment, it generates a target storage policy.
[0085] Scenario 2: When the target storage priority is neither the highest nor the lowest among multiple preset storage priorities, the hardware screening indication information includes a first computing resource load threshold, a second storage type, and a first wear index value range. The hardware configuration information includes the computing resource load index values of multiple memory access nodes and the storage type, address information, and wear information of multiple storage devices. The first computing resource load threshold can be a memory utilization threshold. For example, the second storage type can be a QLC SSD, the memory utilization threshold can be 60%, and the first wear index value range can be between 0.3 and 0.7. Storing code shards with intermediate storage priorities in QLC storage devices can leverage the high-performance storage characteristics of QLC storage devices while reducing wear, thus extending the hardware replacement cycle and reducing maintenance costs.
[0086] The server can determine the first memory access node from multiple memory access nodes based on the first computing resource load threshold and the load index values corresponding to each memory access node. Based on the second storage type, the first wear index value range, and the storage type, address information, and wear information of multiple storage devices, the server determines the first address information and identifies the first storage device corresponding to the first address information from among the multiple storage devices. The server obtains the identification information of the first memory access node and the storage path of the first storage device. Based on the identification information of the target code fragment, the identification information of the first memory access node, the storage path of the first storage device, the first address information, and the target storage priority, the server generates a target storage policy.
[0087] Scenario 3: When the target storage priority is the highest among multiple preset storage priorities, the hardware filtering indication information includes a second compute resource load threshold, a storage resource load threshold, a third storage type, and a second wear indicator value range. The hardware configuration information includes compute resource load indicator values for multiple memory access nodes, load indicator values for multiple storage controllers, and storage type, address information, and wear information for multiple storage devices. The second compute resource load threshold can be the core load utilization rate. For example, the third storage type can be NVME, the core load utilization rate threshold can be 50%, the second wear indicator value range can be less than 0.3, and the storage resource load threshold can be an IOPS threshold, specifically 70%.
[0088] The server can determine the second memory access node from among multiple memory access nodes based on the second computing resource load threshold and the load index values of multiple memory access nodes. Based on the third storage type, storage resource load threshold, second wear index value range, load index values of multiple storage controllers, and storage type, address information, and wear information of multiple storage devices, the server determines the second address information and identifies the second storage device corresponding to the second address information from among the multiple storage devices. The server obtains the identification information of the second memory access node and the storage path of the second storage device. Based on the identification information of the target code fragment, the identification information of the second memory access node, the storage path of the second storage device, the second address information, and the target storage priority, the server generates a target storage policy.
[0089] In this way, for higher storage priorities, binding memory access nodes, storage controllers, and storage devices can improve access performance, while for lower storage priorities, only the storage device needs to be determined, reducing server overhead. Therefore, using different hardware filtering indicators for different storage priorities offers greater flexibility and meets the actual needs of code sharding.
[0090] For example, the target storage strategy can be as follows:
[0091] {
[0092] "chunkId": "chunk-123",
[0093] "priority": "medium",
[0094] "targetDevice": " / dev / nvme0n1p2",
[0095] "numaNode": 1,
[0096] "blockRange": [1024, 2048],
[0097] "expireTime": "2024-12-31T23:59:59Z"
[0098] }
[0099] Among them, "chunk-123" is the identification information of the target code fragment, "medium" is the target storage priority, " / dev / nvme0n1p2" is the storage path of the storage device, [1024,2048] is the address information, and "2024-12-31T23:59:59Z" is the timestamp, which can be used to identify the generation time of the target storage policy.
[0100] In some alternative implementations, in scenario three above, the server may specifically determine the second memory access node using the following steps:
[0101] Step 1: Based on the second computing resource load threshold and the load index values corresponding to the multiple memory access nodes, select at least one candidate memory access node whose load index value is less than the second computing resource load threshold from among the multiple memory access nodes.
[0102] Step 2: Obtain the matching degree between each candidate memory access node and the target code fragment in at least one candidate memory access node.
[0103] Step 3: Based on the matching degree between the target code fragment and each memory access node, determine the candidate memory access node with the highest matching degree among at least one candidate memory access node.
[0104] Step 4: Determine the second memory access node from the candidate memory access node with the highest matching degree.
[0105] Specifically, the server can first compare the second computing resource load threshold with the load metric values corresponding to multiple memory access nodes, and select at least one candidate memory access node whose load metric value is less than the second computing resource load threshold. Then, the server can determine the matching degree between the target code shard and each candidate memory access node, and finally select the candidate memory access node with the highest matching degree as the second memory access node.
[0106] The server can obtain the matching degree between the target code shard and the target candidate memory access node (any candidate memory access node) in the following way:
[0107] Obtain the average dependency distance corresponding to the target candidate memory access node. Based on the average dependency distance, the memory usage and total memory capacity of the target candidate memory access node included in the hardware configuration information, determine the matching degree between the target code fragment and the target candidate memory access node.
[0108] For example, the matching degree between the target code fragment and the target candidate memory access node can be determined by the following expression based on the average dependency distance, the memory usage of the target candidate memory access node included in the hardware configuration information, and the total memory capacity:
[0109] Matching degree = k4 × (1 / average dependency distance) + k5 × (memory usage / total memory capacity) (3)
[0110] Where k4 can be 0.7 and k5 can be 0.3. The average dependency distance is the average physical distance between the memory access nodes of other code fragments that depend on the target code fragment and the target candidate memory access node. For example, the physical distance between two adjacent memory access nodes can be 1, and the physical distance between two memory access nodes can be 2.
[0111] This allows for precise binding between code shards and memory access nodes, reducing cross-node access latency.
[0112] In some alternative implementations, the first memory access node can also be determined in a manner similar to that in Case 3, which will not be elaborated here.
[0113] In some alternative implementations, wear information may include the wear levels of multiple free storage blocks. In scenario two above, the server may determine the first address information and the first storage device using the following specific steps:
[0114] Step 1: Based on the second storage type and the storage types of multiple storage devices, identify candidate storage devices that match the second storage type from among the multiple storage devices.
[0115] Step 2: Based on the address information and wear information of the candidate storage devices, as well as the range of the first wear index value, determine the first address information and the first storage device corresponding to the first address information.
[0116] Specifically, the server can first identify candidate storage devices of the second storage type from multiple storage devices. Then, based on the wear levels of multiple free storage blocks included in the wear information of the candidate storage devices, it can identify candidate free storage blocks within the first wear index value range from the free storage blocks of the candidate storage devices. Next, based on the size of the target code fragment and the address information of the candidate free storage blocks, it can determine the first address information (for example, if the size of the free storage block is a preset length, the server can determine the ratio between the size of the target code fragment and the preset length, that is, determine the required number of free storage blocks. Based on the number of free storage blocks, it can select an equal number of free storage blocks to be written from the candidate free storage blocks and obtain the address information of the free storage blocks to be written as the first address information). Finally, it can determine the first storage device to which the free storage block corresponding to the first address information belongs.
[0117] In some alternative implementations, in scenario three above, the server may determine the second address information and the second storage device using the following specific steps:
[0118] Step 1: Based on the third storage type and the storage types of multiple storage devices, identify candidate storage devices that match the third storage type from among the multiple storage devices.
[0119] Step 2: Based on the load index value of the storage controller connected to the candidate storage device, the storage resource load threshold, the address information and wear information of the candidate storage device, and the range of the second wear index value, determine the second address information and the second storage device corresponding to the second address information.
[0120] Specifically, the server can first identify candidate storage devices of a third storage type from multiple storage devices. Then, the server can identify candidate storage controllers whose load index values are less than the storage resource load threshold from among the storage controllers connected to each candidate storage device. Further, the server can identify candidate free storage blocks whose wear levels fall within the range of a second wear index value from among the free storage blocks included in the wear information of the candidate storage devices connected to the candidate storage controllers, based on the wear levels of multiple free storage blocks. The server can determine second address information based on the size of the target code fragment and the address information of the candidate free storage blocks (for example, if the free storage blocks are of a consistent size, which can be a preset length, the ratio between the size of the target code fragment and the preset length can be determined, i.e., the required number of free storage blocks can be determined; based on the number of free storage blocks, an equal number of free storage blocks to be written can be selected from the candidate free storage blocks, and the address information of the free storage blocks to be written can be obtained as the second address information). Finally, the second storage device to which the free storage block corresponding to the second address information belongs can be determined.
[0121] In some optional implementations, in step S204 above, the server may perform storage operations on the target code fragments using the following specific steps:
[0122] Step 1: Based on the identification information of the target memory access node included in the target storage strategy, obtain the core identification information corresponding to the target memory access node.
[0123] The target memory access node can be either the first memory access node or the second memory access node.
[0124] Step 3: Create a loading process for the target code fragment.
[0125] Step four: Based on the core identification information, bind the loading process to the core of the target memory access node so that the loading process can run in the core of the target memory access node.
[0126] Step 5: Using the loading process, load the target code fragments into the target memory of the target memory access node.
[0127] Step 6: Based on the storage path and address information of the storage devices included in the target storage strategy, perform storage operations on the target code fragments loaded into the target memory.
[0128] Specifically, the server can obtain the core list of the target memory access node based on the target memory access node's identification information and a preset acquisition command. The core list includes the core identification information of each core in the target memory access node. For example, the preset acquisition command could be "cat / sys / devices / system / node / node0 / cpulist". Then, the server can start the target code fragment loading process (chunk_loader), using the preset "pthread_setaffinity_np function" and the core identification information to bind the loading process to the core of the target memory access node, ensuring that the loading process runs within the core of the target memory access node and preventing process scheduling from crossing nodes. The server can call the "set_mempolicy function" and input the memory allocation policy as "MPOL_BIND" to instruct the allocation of physical memory from the target memory access node to the target code fragment. After completing the physical memory allocation operation, the server can use a pre-built memory map (MMAP) function mapping file to load the target code fragment into the target memory of the target memory access node.
[0129] Before loading, the server can also input "MADV_HUGEPAGE" into the Memory Advice (MADVISE) function to prompt the server's operating system kernel to use 2MB big pages instead of the default 4KB small pages, which can improve memory access efficiency.
[0130] When the server receives a storage instruction corresponding to a target code fragment, it can identify the target storage strategy for the target code fragment based on its identifier information. Then, based on the storage path and address information recorded in the target storage strategy, it stores the target code fragment in target memory into a storage device that matches both the storage path and address information. For example, the target code fragment can be stored in the storage area corresponding to the first address information in a first storage device, or it can be stored in the storage area corresponding to the second address information in a second storage device.
[0131] In some alternative implementations, the server may periodically perform memory page offset detection operations on the target memory access node, and the specific steps may include:
[0132] The server can periodically extract data from " / proc / <pid>The system reads the number of memory pages in the target memory access node and other memory access nodes from the loading process in " / numa_maps". Based on the number of memory pages in the target memory access node and other memory access nodes in two adjacent cycles, it determines the drift rate of the target memory access node. When the drift rate is determined to be greater than a preset drift rate threshold, the memory binding (MBIND) function is called to rebind the memory pages and record the reason for the drift (e.g., memory fragmentation, kernel memory reclamation, etc.).
[0133] For example, when it is determined that the drift rate is >5% (that is, 50 out of 1000 memory pages of the target memory access node drift to other memory access nodes), the memory binding function is called to rebind the memory pages.
[0134] In some optional implementations, when it is determined that a new code fragment has been added or some code fragments have been modified in the target code file, the server can recalculate the matching degree between each code fragment and its current memory access node. Taking the target code fragment as an example, when the difference between the new matching degree of the target code fragment and the original matching degree is greater than a preset difference (for example, it can be 20%), the matching degree of the target code fragment with other memory access nodes can be recalculated, and the memory access node with the highest matching degree can be selected from the other memory access nodes. The target code fragment is then migrated to the memory access node with the highest matching degree.
[0135] In this way, the memory access nodes bound to the target code shards can be adjusted at any time as the situation changes in real time, resulting in greater flexibility.
[0136] In some alternative implementations, in scenario two above, the server may also select candidate storage blocks in the following manner:
[0137] The server can determine the wear level type of each free storage block based on its wear level. Additionally, the server can obtain the status (free or occupied) and last write time of each free storage block, and construct a mapping table based on the free storage block's identification information, wear level type, status, and last write time, storing it in the non-volatile memory of the baseboard management controller. For example, the wear level type can include low wear area, medium wear area, and high wear area. The low wear area type corresponds to a wear level range of "less than 0.3", the medium wear area type corresponds to a wear level range of "0.3-0.7", and the high wear area type corresponds to a wear level range of "greater than 0.7 and less than or equal to 1".
[0138] When the server receives a write request for the target code shard, it can retrieve the historical write records of the target code shard based on its identifier information. These historical write records can include write timestamps of the target code shard within a preset time period (e.g., 30 days). The server can then determine the number of write timestamps recorded each day as the number of writes to the target code shard on that day. Next, the server can input the number of writes for each day within the preset time period into a pre-built write count prediction model to obtain the predicted write count output by the model (which could be the average daily write count for the next 7 days). The write count prediction model can be an Autoregressive Integrated Moving Average (ARIMA), whose parameters include the order of the autoregressive term (which can be 2), the order of the differencing term (which can be 1), and the order of the moving average term (which can be 1).
[0139] The server determines the predicted write count range based on the predicted write count, and then identifies the wear zone type corresponding to this range as the target wear zone type for the target code shard. Additionally, the server can determine the wear zone type for each free storage block based on the wear level of the free storage blocks included in the hardware configuration information. For example, a write count range >100 times / day corresponds to a low wear zone type, a write count range of 10-100 times / day corresponds to a medium wear zone type, and a write count range <10 times / day corresponds to a high wear zone type.
[0140] Accordingly, in the second scenario above, the server can select the free storage block corresponding to the target wear area type as the candidate free storage block based on the target wear area type of the target code fragment and the wear area type of each free storage block.
[0141] In this way, predicting future write counts based on the write frequency of code shards and allocating appropriate storage tasks to different worn blocks based on the write count can extend hardware lifespan.
[0142] In some optional implementations, the server can update the wear zone type of each free storage block in real time based on the wear level of each free storage block. When it is determined that the wear zone type of the free storage block storing the target code fragment is inconsistent with the target wear zone type, the target code fragment can be migrated to a free storage block with the same wear zone type during a preset low-load period (e.g., 3 AM to 5 AM). For example, multi-threading can be started to perform the migration operation. During the migration process, a file send function (e.g., "sendfile") can be used to achieve zero-copy data transfer. After the migration is completed, the inode can be updated to point to the new storage block, and the block discard function (e.g., "blkdiscard") can be called to erase the data in the original storage block and release it to a free state.
[0143] In some alternative implementations, the server's browser can load the front-end page via " <script type="module">”导入WASM模块(文件名称可以是"nvme_driver.wasm”),同时创建2MB大小的"Shared Array Buffer”作为数据传输缓冲区(该数据传输缓冲区可以配置为COOP或COEP头启用跨域共享,后续可以将该数据传输缓冲区称为共享内存区),以便JavaScript代码和WASM模块可以直接读写同一块内存,使得JavaScript代码可以调用WASM模块导出的函数,WASM模块也可以通过回调函数调用JavaScript代码。相应地,在需要将目标代码分片写入到NVME类型的存储设备时,服务器可以通过JavaScript代码调用WASM模块的"nvme_write函数”,并传入的相关参数,例如,相关参数可以包括共享内存区指针(也即目标代码分片在共享内存区中的偏移量)、目标代码分片的大小(≤16MB)、目标存储策略中的地址信息、签名信息(可以基于数据哈希和WASM模块的设备证书生成)。进一步,服务器可以利用WASM模块基于传入的相关参数构造请求结构体(请求结构体还可以包括用于指示写操作的操作码,例如,"0x01”),然后,通过"vmmcall指令”触发虚拟机监控器(VirtualMachine Monitor,VMM)拦截请求结构体。VMM拦截该请求结构体后,对请求结构体执行安全验证,例如,可以采用BMC预存公钥比对校验签名信息的有效性、验证共享内存区指针指示的地址是否在共享内存区范围内,以防止越界访问、检查地址信息是否目标存储策略允许的地址范围内。
[0144] 在验证通过后,VMM可以调用NVME驱动的"submit_io接口”,将共享内存区的物理地址填入物理区域页(Physical Region Page,PRP)列表,并基于请求结构体,生成IO请求,提交IO请求至NVME控制器。NVME控制器可以根据IO请求,从共享内存区中读取目标代码分片,存入到对应的存储设备中,也即完成直接内存访问(Direct Memory Access,DMA)传输。在完成传输后,通过中断通知VMM,VMM将结果(成功或失败、实际写入长度)写入共享内存区的状态区(状态区的偏移量可以是0x00)。WASM模块轮询状态区(例如,间隔10μs)获取写操作结果,通过JavaScript代码回调通知浏览器写入完成。
[0145] 在当前的架构中,前端代码与存储设备之间的数据传输需要经过操作系统的多层调度和处理(如文件系统、驱动程序等),这种间接访问方式引入了额外的延迟和性能开销,无法满足对响应时间要求极高的低延迟应用场景(如实时视频处理、自动驾驶、工业控制等),而本方案可以绕过操作系统调度,实现浏览器WASM模块到NVME控制器的零拷贝数据传输,大幅降低延迟。
[0146] 在一些可选的实施方式中,在上述迁移目标代码分片的过程中,服务器可以首先读取目标代码分片的前16MB的内容,并启动GPU执行校验任务,具体来说,GPU可以通过"CUDA Runtime API”分配显存,使得服务器可以将目标代码分片拷贝到分配的显存中。进而,GPU可以启动"crc64_kernel核函数”,调用多个线程并行计算前16MB内容的源循环冗余校验(Cyclic Redundancy Check,CRC)和,并将源CRC和存储到预先指定的源数据(也即目标代码分片)的元信息区。其中,每一个线程可以处理其中的256字节的数据。然后,GPU可以通过远程直接内存访问(Remote Direct Memory Access,RDMA)协议将目标代码分片传输至目的存储设备(前述确定出的与目标存储类型一致的存储设备)。在传输目标代码分片的过程中,FPGA可以通过PCIE DMA监控数据帧,提取源CRC和。目的存储设备接收完成后,再重复上述的GPU校验过程,得到目标CRC和。如果FPGA判断源CRC和与目标CRC和一致,则可以确认迁移成功(例如,可以向硬件感知策略生成器发送迁移成功信号,用于硬件感知策略生成器对目标存储策略中的地址信息和存储路径进行更新),如果不一致,则可以触发异常处理。例如,FPGA可以触发拉低回滚控制引脚,使得FPGA可以从目标存储器中加载预存比特流(可以是目的存储设备中原数据的快照),将目的存储设备中的数据恢复至迁移前的状态。其中,目标存储器可以是电可擦除可编程只读存储器(Electrically ErasableProgrammable Read-Only Memory,EEPROM)。同时,FPGA可以记录错误日志,用于后续故障分析。
[0147] 这样,利用GPU并行计算加速校验,FPGA实现实时监控与快速回滚,保障数据迁移一致性,并且效率较高。
[0148] 在一些可选的实施方式中,如图3所示,CPU可以用于运行抽象语法树(AbstractSyntax Tree,AST)拓扑分析引擎和Webpack / Vite插件。Webpack / Vite插件可以用于对目标代码文件进行解析,生成各个代码分片。AST拓扑分析引擎可以用于对组成目标代码文件的各个代码分片进行AST分析,获取到各个代码分片的代码结构信息,以及基于代码结构信息,确定出异步操作密度、引用中心度、性能类型标记,进而,可以通过前端构建层将目标代码分片的标识信息、异步操作密度、引用中心度、性能类型标记封装为特征对象,并通过emit钩子函数,将代码分片和特征对象输出至协同控制层。这样,emit钩子函数可以对发送的代码分片和特征对象进行商用密码算法-分组密码算法第4号(SM4)加密,得到加密内容,将加密内容通过超文本传输协议(Hyper Text Transfer Protocol,HTTP) / 2安全传输至协同控制层。其中,AST拓扑分析引擎、Webpack / Vite插件共同构成了前端构建层。
[0149] 另外,CPU还可以用于运行硬件感知策略生成器,并启动迁移验证引擎。迁移验证引擎包括上述的GPU和FPGA。硬件感知策略生成器、GPU和FPGA共同构成了协同控制层。硬件感知策略生成器可以用于生成目标存储策略,并远程过程调用(Remote Procedure Call,RPC)将目标存储策略推送至存储层。
[0150] 存储控制器和存储设备共同构成了存储层。存储控制器可以用于监测存储设备中包括的各个存储块的磨损水平。每一个内存访问节点用于管理与自身对应的存储控制器和存储设备。例如,目标代码分片对应的内存访问节点可以用于管理目标代码分片存储的存储设备及其对应的存储控制器。管理操作可以包括从存储控制器读取存储设备包括的各个存储块的磨损水平。内存访问节点中的CPU可以运行内存访问节点绑定控制器,用于绑定特定线程或进程,例如,上述绑定加噪进程的操作。BMC可以运行寿命管理器,寿命管理器可以用于获取目标代码分片的历史写入记录,以及各个存储块的磨损水平,并采用ARIMA模型预测目标代码分片的预测写入次数。
[0151] 例如,特征对象可以是JavaScript对象表示法(JavaScript Object Notation,JSON)对象:
[0152] {"chunkId":"chunk-123","topologyFeatures":{"asyncDensity":0.25,"depCentrality":0.8,"wasmUsage":true}}
[0153] 其中,"chunk-123"为目标代码分片的标识信息,"asyncDensity":0.25为异步操作密度,"depCentrality":0.8为引用中心度,"wasmUsage":true为性能类型标记。
[0154] 在一些可选的实施方式中,服务器可以启动策略计算主进程和硬件采集子进程。策略计算主进程可以用于生成存储策略,硬件采集子进程可以用于周期性采集硬件配置信息。服务器可以设置有环形缓冲区,用于存储硬件配置信息。这样,主进程崩溃后,仍可通过子进程采集数据。
[0155] 通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到根据上述实施例的方法可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件,但很多情况下前者是更佳的实施方式。
[0156] 本申请的实施例还提供了一种代码分片的存储装置,如图4所示,包括:
[0157] 获取模块410,用于获取目标代码分片的标识信息和代码结构信息、目标代码文件中除目标代码分片之外的其他代码分片与目标代码分片之间的引用关系信息,以及服务器的硬件配置信息;
[0158] 确定模块420,用于根据代码结构信息和引用关系信息,确定目标代码分片的目标存储优先级;
[0159] 生成模块430,用于根据目标代码分片的标识信息、硬件配置信息,以及目标存储优先级,生成目标代码分片的目标存储策略;
[0160] 存储模块440,用于采用目标存储策略,对目标代码分片执行存储操作。
[0161] 在一些可选的实施方式中,确定模块420,具体用于:
[0162] 根据代码结构信息和引用关系信息,确定目标代码分片的存储权重值;
[0163] 根据存储权重值,在多个预设的权重值区间中确定出存储权重值所属的目标权重值区间;
[0164] 将与目标权重值区间对应的存储优先级确定为目标存储优先级。
[0165] 在一些可选的实施方式中,代码结构信息包括异步代码节点的数量、有效代码行数,以及性能类型标记,引用关系信息包括目标代码分片被其他代码分片引用的第一数量和目标代码分片引用其他代码分片的第二数量;
[0166] 确定模块420,具体用于:
[0167] 根据异步代码节点的数量和有效代码行数,确定目标代码分片的异步操作密度,其中,异步代码节点为在代码执行过程中会导致当前执行流停止,直至获取到目标代码节点反馈的结果后才启动当前执行流的代码节点,目标代码节点为目标代码文件中的任一个代码分片中的代码节点;
[0168] 根据第一数量和第二数量,确定目标代码分片在目标代码文件中的重要程度指标值;
[0169] 根据性能类型标记、异步操作密度,以及重要程度指标值,确定存储权重值。
[0170] 在一些可选的实施方式中,生成模块430,具体用于:
[0171] 根据目标存储优先级,获取与目标存储优先级对应的硬件筛选指示信息;
[0172] 根据目标代码分片的标识信息、硬件筛选指示信息,以及硬件配置信息,生成目标存储策略。
[0173] 在一些可选的实施方式中,当目标存储优先级为多个预设存储优先级中最低的存储优先级时,硬件筛选指示信息包括第一存储类型,硬件配置信息包括多个存储设备的存储类型;
[0174] 生成模块430,具体用于:
[0175] 根据第一存储类型和至少一个存储设备的存储类型,在多个存储设备中确定出与第一存储类型对应的存储设备;
[0176] 获取与第一存储类型对应的存储设备的存储路径;
[0177] 根据与第一存储类型对应的存储设备的存储路径和目标代码分片的标识信息,生成目标存储策略。
[0178] 在一些可选的实施方式中,当目标存储优先级并非为多个预设存储优先级中最高的存储优先级,且并非为多个预设存储优先级中最低的存储优先级时,硬件筛选指示信息包括第一计算资源负载阈值、第二存储类型,以及第一磨损指标值范围,硬件配置信息包括多个内存访问节点的计算资源负载指标值和多个存储设备的存储类型、地址信息和磨损信息;
[0179] 生成模块430,具体用于:
[0180] 根据第一计算资源负载阈值和多个内存访问节点分别对应的负载指标值,在多个内存访问节点中确定出第一内存访问节点;
[0181] 根据第二存储类型、第一磨损指标值范围,以及多个存储设备的存储类型、地址信息和磨损信息,确定第一地址信息,并在多个存储设备中确定出与第一地址信息对应的第一存储设备;
[0182] 获取第一内存访问节点的标识信息和第一存储设备的存储路径;
[0183] 根据目标代码分片的标识信息、第一内存访问节点的标识信息、第一存储设备的存储路径、第一地址信息,以及目标存储优先级,生成目标存储策略。
[0184] 在一些可选的实施方式中,当目标存储优先级为多个预设存储优先级中最高的存储优先级时,硬件筛选指示信息包括第二计算资源负载阈值、存储资源负载阈值、第三存储类型,以及第二磨损指标值范围,硬件配置信息包括多个内存访问节点的计算资源负载指标值、多个存储控制器的负载指标值,以及多个存储设备的存储类型、地址信息和磨损信息;
[0185] 生成模块430,具体用于:
[0186] 根据第二计算资源负载阈值和多个内存访问节点的负载指标值,在多个内存访问节点中确定出第二内存访问节点;
[0187] 根据第三存储类型、存储资源负载阈值、第二磨损指标值范围、多个存储控制器的负载指标值,以及多个存储设备的存储类型、地址信息和磨损信息,确定第二地址信息,并在多个存储设备中确定与第二地址信息对应的第二存储设备;
[0188] 获取第二内存访问节点的标识信息和第二存储设备的存储路径;
[0189] 根据目标代码分片的标识信息、第二内存访问节点的标识信息、第二存储设备的存储路径、第二地址信息,以及目标存储优先级,生成目标存储策略。
[0190] 在一些可选的实施方式中,生成模块430,具体用于:
[0191] 根据第二计算资源负载阈值和多个内存访问节点分别对应的负载指标值,在多个内存访问节点中选取负载指标值小于第二计算资源负载阈值的至少一个候选内存访问节点;
[0192] 获取至少一个候选内存访问节点中每一个候选内存访问节点与目标代码分片之间的匹配度;
[0193] 根据目标代码分片与每一个内存访问节点之间的匹配度,在至少一个候选内存访问节点中确定出匹配度最大的候选内存访问节点;
[0194] 将匹配度最大的候选内存访问节点确定第二内存访问节点。
[0195] 在一些可选的实施方式中,存储模块440,具体用于:
[0196] 根据目标存储策略包括的目标内存访问节点的标识信息,获取与目标内存访问节点对应的核心标识信息;
[0197] 创建目标代码分片的加载进程;
[0198] 根据核心标识信息,将加载进程与目标内存访问节点的核心进行绑定,以便在目标内存访问节点的核心中运行加载进程;
[0199] 利用加载进程,将目标代码分片加载至目标内存访问节点中目标内存中;
[0200] 根据目标存储策略中包括的存储设备的存储路径和地址信息,对加载至目标内存中的目标代码分片执行存储操作。
[0201] 代码分片的存储装置所对应实施例中特征的说明可以参见代码分片的存储方法所对应实施例的相关说明,这里不再一一赘述。
[0202] 本申请的实施例还提供了一种电子设备,如图5所示,包括存储器10和处理器20,该存储器10中存储有计算机程序,该处理器20被设置为运行计算机程序以执行上述任一种代码分片的存储方法实施例中的步骤。该电子设备可以是上述的服务器。
[0203] 本申请的实施例还提供了一种计算机可读存储介质,该计算机可读存储介质中存储有计算机程序,其中,该计算机程序被设置为运行时执行上述任一种代码分片的存储方法实施例中的步骤。
[0204] 在一个示例性实施例中,上述计算机可读存储介质可以包括但不限于:U盘、只读存储器(Read-Only Memory,简称为ROM)、随机存取存储器(Random Access Memory,简称为RAM)、移动硬盘、磁碟或者光盘等各种可以存储计算机程序的介质。
[0205] 本申请的实施例还提供了一种计算机程序产品,上述计算机程序产品包括计算机程序,计算机程序被处理器执行时实现上述任一种代码分片的存储方法实施例中的步骤。
[0206] 本申请的实施例还提供了另一种计算机程序产品,包括非易失性计算机可读存储介质,非易失性计算机可读存储介质存储计算机程序,计算机程序被处理器执行时实现上述任一种代码分片的存储方法实施例中的步骤。
[0207] 专业人员还可以进一步意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、计算机软件或者二者的结合来实现,为了清楚地说明硬件和软件的可互换性,在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。
[0208] 以上对本申请所提供的一种代码分片的存储方法、装置、电子设备、存储介质及程序产品进行了详细介绍。本文中应用了具体个例对本申请的原理及实施方式进行了阐述,以上实施例的说明只是用于帮助理解本申请的方法及其核心思想。应当指出,对于本技术领域的普通技术人员来说,在不脱离本申请原理的前提下,还可以对本申请进行若干改进和修饰,这些改进和修饰也落入本申请的保护范围内。< / script> < / pid>
Claims
1. A method for storing code fragments, characterized in that, include: The system obtains the identification information and code structure information of the target code segment, the reference relationship information between the target code segment and other code segments in the target code file besides the target code segment, and the hardware configuration information of the server. The code structure information includes the number of asynchronous code nodes, the number of effective code lines, and the performance type flag. The reference relationship information includes a first number of references of the target code segment by other code segments and a second number of references of the target code segment to other code segments. The asynchronous operation density of the target code segment is determined based on the number of asynchronous code nodes and the number of effective lines of code. The asynchronous code node is a code node that causes the current execution flow to stop during code execution and only starts the current execution flow after obtaining the result fed back by the target code node. The target code node is a code node in a code segment of the target code file. Based on the first quantity and the second quantity, determine the importance index value of the target code fragment in the target code file; The storage weight value is determined based on the performance type label, the asynchronous operation density, and the importance index value. Based on the stored weight value, the target weight value range to which the stored weight value belongs is determined among multiple preset weight value ranges; The storage priority corresponding to the target weight value range is determined as the target storage priority; Based on the identifier information of the target code fragment, the hardware configuration information, and the target storage priority, a target storage strategy for the target code fragment is generated. The target code fragments are stored using the target storage strategy.
2. The code fragment storage method according to claim 1, characterized in that, The step of generating a target storage strategy for the target code fragment based on the identifier information of the target code fragment, the hardware configuration information, and the target storage priority includes: Based on the target storage priority, obtain the hardware filtering indication information corresponding to the target storage priority; The target storage strategy is generated based on the identifier information of the target code fragment, the hardware filtering instruction information, and the hardware configuration information.
3. The code fragment storage method according to claim 2, characterized in that, When the target storage priority is the lowest among multiple preset storage priorities, the hardware filtering indication information includes a first storage type, and the hardware configuration information includes the storage types of multiple storage devices; The step of generating the target storage strategy based on the identifier information of the target code fragment, the hardware filtering indication information, and the hardware configuration information includes: Based on the first storage type and the storage type of at least one of the storage devices, determine the storage device corresponding to the first storage type from among the plurality of storage devices; Obtain the storage path of the storage device corresponding to the first storage type; The target storage strategy is generated based on the storage path of the storage device corresponding to the first storage type and the identification information of the target code fragment.
4. The code fragment storage method according to claim 2, characterized in that, When the target storage priority is neither the highest nor the lowest among the preset storage priorities, the hardware filtering indication information includes a first computing resource load threshold, a second storage type, and a first wear index value range. The hardware configuration information includes the computing resource load index values of multiple memory access nodes and the storage type, address information, and wear information of multiple storage devices. The step of generating the target storage strategy based on the identifier information of the target code fragment, the hardware filtering indication information, and the hardware configuration information includes: Based on the first computing resource load threshold and the load index values corresponding to the plurality of memory access nodes, the first memory access node is determined from the plurality of memory access nodes; Based on the second storage type, the first wear index value range, and the storage type, address information, and wear information of the multiple storage devices, the first address information is determined, and the first storage device corresponding to the first address information is determined among the multiple storage devices; Obtain the identification information of the first memory access node and the storage path of the first storage device; The target storage strategy is generated based on the identification information of the target code fragment, the identification information of the first memory access node, the storage path of the first storage device, the first address information, and the target storage priority.
5. The code fragment storage method according to claim 2, characterized in that, When the target storage priority is the highest among multiple preset storage priorities, the hardware filtering indication information includes a second computing resource load threshold, a storage resource load threshold, a third storage type, and a second wear index value range. The hardware configuration information includes the computing resource load index values of multiple memory access nodes, the load index values of multiple storage controllers, and the storage type, address information, and wear information of multiple storage devices. The step of generating the target storage strategy based on the identifier information of the target code fragment, the hardware filtering indication information, and the hardware configuration information includes: Based on the second computing resource load threshold and the load index values of the multiple memory access nodes, the second memory access node is determined among the multiple memory access nodes; Based on the third storage type, the storage resource load threshold, the range of the second wear index value, the load index values of multiple storage controllers, and the storage type, address information, and wear information of multiple storage devices, the second address information is determined, and the second storage device corresponding to the second address information is determined among the multiple storage devices. Obtain the identification information of the second memory access node and the storage path of the second storage device; The target storage strategy is generated based on the identification information of the target code fragment, the identification information of the second memory access node, the storage path of the second storage device, the second address information, and the target storage priority.
6. The code fragment storage method according to claim 5, characterized in that, The step of determining the second memory access node from among the multiple memory access nodes based on the second computing resource load threshold and the load index values of the multiple memory access nodes includes: Based on the second computing resource load threshold and the load index values corresponding to the plurality of memory access nodes, at least one candidate memory access node whose load index value is less than the second computing resource load threshold is selected from the plurality of memory access nodes. Obtain the matching degree between each of the candidate memory access nodes and the target code segment; Based on the matching degree between the target code fragment and each memory access node, the candidate memory access node with the highest matching degree is determined among at least one candidate memory access node; The candidate memory access node with the highest matching degree is determined as the second memory access node.
7. The code fragment storage method according to any one of claims 4 to 6, characterized in that, The step of storing the target code fragments using the target storage strategy includes: Based on the identification information of the target memory access node included in the target storage strategy, obtain the core identification information corresponding to the target memory access node; Create a loading process for the target code fragment; Based on the core identification information, the loading process is bound to the core of the target memory access node so that the loading process can run in the core of the target memory access node; Using the loading process, the target code fragments are loaded into the target memory of the target memory access node; Based on the storage path and address information of the storage device included in the target storage strategy, storage operations are performed on the target code fragments loaded into the target memory.
8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the storage method for code fragmentation as described in any one of claims 1 to 7 when executing the computer program.
Citation Information
Patent Citations
Resource processing method and device, storage medium and cache server
CN112433848A