Multi-dimensional flight data hierarchical storage management method, system, device and storage medium

Abstract

The present application relates to the technical field of unmanned aerial vehicle, and specifically provides a multi-dimensional flight data hierarchical storage management method, system, device and storage medium, comprising: firstly, real-time monitoring of hierarchical storage system resource state, when the node meets the downgrade trigger condition, determining the source node to be downgraded; based on the access frequency, value and data classification label of data fragmentation, sensitive level and compliance rules, screening out target downgrade data. Further, planning a downgrade path for it: first, excluding non-compliant paths according to compliance rules to obtain a candidate set, and then using a downgrade decision model, combining the real-time state of source and target nodes, data attributes and migration cost benefit model, determining the optimal downgrade path from the candidate set. Finally, migrating data according to the path and updating the access route, so as to ensure that the downgrade decision can dynamically adapt to the rapid conversion of unmanned aerial vehicle tasks.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) technology, specifically relating to a multi-dimensional flight data hierarchical storage management method, system, device, and storage medium. Background Technology

[0002] With the widespread application of drone technology in surveying, inspection, and security, the data generated by their onboard sensors is growing rapidly in massive volumes. This data exhibits distinct hierarchical value characteristics: real-time imagery and sensing data during critical mission phases require millisecond-level access and are considered high-value, hot data; while the access frequency of large-volume raw videos and historical logs drops sharply after the mission, transforming them into low-value, cold data. To ensure the real-time performance of critical missions and optimize overall storage costs within extremely limited high-performance storage resources onboard or edge devices, adopting a tiered storage system has become an inevitable choice for the industry. This involves migrating and managing data based on its dynamic value across different storage tiers with varying performance, capacity, and cost (such as high-speed flash memory, hybrid storage, high-density hard disk drives, and archive media).

[0003] However, existing tiered storage management methods applied to drones or similar resource-constrained environments primarily follow the traditional strategies based on static rules and thresholds used in data centers. These methods typically pre-set simple triggering conditions (such as storage utilization exceeding a fixed threshold in a certain tier) and employ a single, fixed degradation path (e.g., only allowing migration from the highest-performing tier to the next adjacent tier). In the specific application scenario of drones, these traditional methods expose the following core technical problems: First, there's the issue of rigid decision-making and scenario mismatch. Drone missions are highly phased and scenario-dependent, with the popularity and value of data rapidly changing with mission phases (such as reconnaissance, tracking, and return-to-home analysis). Static rules cannot perceive the mission context and rely solely on storage usage as a single dimension for decision-making. This makes it highly susceptible to mistakenly degrading valuable data or data about to be accessed under storage pressure, leading to performance degradation for critical missions and an inability to adaptively adjust strategies during mission transitions.

[0004] Secondly, there is the problem of inefficient resource release. In order to quickly free up valuable high-speed storage space to meet new tasks, it is often necessary to urgently downgrade large-capacity cold data (such as video backups that have already been processed). The traditional "step-by-step degradation" path is lengthy, with huge total migration time and system overhead, which cannot meet the timeliness requirements of emergency space reclamation and lacks a mechanism to seek the globally optimal degradation path.

[0005] Secondly, there is a lack of multi-objective coordination and dynamic trade-offs. Drone storage management requires the simultaneous optimization of multiple often conflicting objectives, such as space release, real-time performance impact, hardware lifespan degradation, system energy consumption, and data security compliance. Traditional methods rely on pre-setting weights based on human experience, which cannot achieve intelligent multi-objective Pareto optimality in dynamic operation, and is even less able to cope with the dynamic changes in priority objectives under different task modes.

[0006] Finally, there are system stability issues caused by passive response. Threshold-based passive response mechanisms are prone to triggering large-scale, centralized degradation operations ("avalanche effect") at capacity critical points, causing significant I / O resource contention and performance jitter, interfering with the execution of real-time tasks in the foreground.

[0007] Therefore, designing an adaptive hierarchical storage management method that can accurately identify degraded objects, dynamically plan the optimal path, intelligently coordinate multiple objectives, and has predictive and smooth execution capabilities in the context of drones with strictly limited resources and dynamically changing scenarios has become an urgent technical challenge. Summary of the Invention

[0008] In view of the above-mentioned shortcomings of the prior art, the present invention provides a multi-dimensional flight data hierarchical storage management method, system, device and storage medium to solve the above-mentioned technical problems.

[0009] In a first aspect, the present invention provides a multi-dimensional flight data hierarchical storage and management method, comprising: Real-time monitoring of the resource status of the pre-built hierarchical storage system; when the status of any storage node meets the preset degradation trigger condition, the storage node is identified as the source node to be degraded; Based on the access frequency, data value, and data classification tags of the data shards in the source node to be downgraded, target downgraded data is selected; the data classification tags are at least associated with data sensitivity levels and storage compliance rules. The process of planning a degradation migration path for the target degradation data includes: eliminating all non-compliant potential degradation paths based on the storage compliance rules associated with the target degradation data to obtain a candidate path set; and using a degradation decision model, determining the optimal degradation path from the candidate path set based on the real-time status of the source node to be downgraded and the downstream target node, the attributes of the target degradation data, and a migration cost-benefit model. According to the optimal degradation path, the target degradation data is migrated to the downstream storage node, and the data access route is updated.

[0010] In one optional implementation, the hierarchical storage system includes: As a primary storage node in the high-speed access layer, it uses flash-based storage media; As a secondary storage node that balances performance and capacity, it uses a hybrid storage medium of solid-state storage and mechanical storage. As a tertiary storage node serving as a high-capacity access layer, it employs high-density mechanical storage media; As a fourth-level storage node for long-term archiving, a tape library or high-capacity sequential access storage medium is used. Among them, the data access performance of storage nodes decreases sequentially from level 1 to level 4, and the unit storage cost decreases sequentially.

[0011] In an optional implementation, the degradation triggering condition includes: The current capacity utilization rate of the storage node exceeds the preset capacity threshold; The current performance metrics of the storage node have degraded by more than a preset amount compared to the baseline value; The performance metrics include at least one of input / output operation count, throughput, or access latency.

[0012] In one optional implementation, target data for degradation is selected based on the access frequency, data value, and data classification tags of the data shards in the source node to be downgraded, including: Based on a pre-built data evaluation model, each data shard in the source node to be downgraded is scored. The input of the data evaluation model includes at least: the historical access frequency of the data shard; the calculated value weight of the data shard, which is determined based on at least one of the importance of the data, the generation cost, or the business relevance; and the data classification label associated with the data shard. Select data segments with scores higher than a preset threshold as the target downgrade data; The data evaluation model includes: Overall score = W1 × visit frequency score + W2 × value weight score + W3 × compliance score; The access frequency score is negatively correlated with the historical access frequency of the data shard; the value weight score is calculated based on the importance, generation cost, or business relevance of the data shard; the compliance score is determined based on the data sensitivity level and retention period requirements indicated by the data classification label; and the weight coefficients W1, W2, and W3 are dynamically adjusted according to the system warning status of the source node to be downgraded.

[0013] In an optional implementation, based on the storage compliance rules associated with the target degraded data, all non-compliant potential degrade paths are excluded to obtain a candidate path set, including: Based on the data classification tags associated with the target downgraded data, obtain the corresponding set of mandatory storage compliance rules; Traverse all potential migration paths from the source node to be degraded to its downstream storage nodes at each level. Potential migration paths include direct migration paths or stepped migration paths. For each of the potential migration paths, verify whether the attributes of its target storage node satisfy each storage compliance rule in the storage compliance rule set; Potential migration paths that meet all storage compliance rules are included in the candidate path set.

[0014] In an optional implementation, a degradation decision model is used to determine the optimal degradation path from the candidate path set based on the real-time status of the source node to be degraded and the downstream target node, the attributes of the target degradation data, and a migration cost-benefit model, including: Construct a state input vector for path evaluation, the state input vector including at least: the real-time performance and capacity status of the source node to be degraded and the downstream target node, the multi-dimensional attributes of the target degradation data, and the pre-evaluation results of the migration cost-benefit model for the candidate path; The state input vector is input into a pre-trained degradation decision model; the degradation decision model is a policy network trained based on deep reinforcement learning; the policy network is trained to maximize long-term rewards, and the long-term rewards are calculated based on the system space state after historical degradation operations, changes in performance indicators, and compliance audit results. The degradation decision model outputs a comprehensive score for each candidate path. The comprehensive score is obtained by weighted calculation of migration cost, space release benefit, performance impact assessment and compliance satisfaction. The candidate path with the highest comprehensive score is selected as the optimal degradation path.

[0015] In an optional implementation, the target degraded data is migrated to the downstream storage node according to the optimal degrade path, and the data access route is updated, including: During the preparation phase, the target degraded data is marked as pending migration, and redirection information pointing to the target downstream storage node is pre-registered in the metadata. During the execution phase, the data content is asynchronously migrated to the downstream storage node determined by the optimal degradation path in the background. During the migration, access requests for the data are transparently redirected to the original or target location through the redirection information. In the completion phase, after verifying the integrity of the target location data, the redirection information in the metadata is updated to permanent routing information, and the original data space in the source storage node is reclaimed.

[0016] Secondly, the present invention provides a multi-dimensional flight data hierarchical storage and management system, comprising: The real-time monitoring module is used to monitor the resource status of the pre-built hierarchical storage system in real time; when the status of any storage node meets the preset degradation trigger condition, the storage node is identified as the source node to be degraded. The target determination module is used to filter out target degraded data based on the access frequency, data value, and data classification tags of the data shards in the source node to be degraded; the data classification tags are at least associated with data sensitivity level and storage compliance rules; The path planning module is used to plan a degradation migration path for the target degradation data. The planning process includes: excluding all non-compliant potential degradation paths according to the storage compliance rules associated with the target degradation data to obtain a candidate path set; and using a degradation decision model, determining the optimal degradation path from the candidate path set based on the real-time status of the source node to be downgraded and the downstream target node, the attributes of the target degradation data, and the migration cost-benefit model. The degradation execution module is used to migrate the target degradation data to the downstream storage node according to the optimal degradation path and update the data access route.

[0017] Thirdly, a device is provided, comprising: The memory is used to store the hierarchical storage management program for multidimensional flight data; A processor is configured to implement the steps of the multidimensional flight data hierarchical storage management method as provided in the first aspect when executing the multidimensional flight data hierarchical storage management program.

[0018] Fourthly, a computer-readable storage medium is provided, on which a multidimensional flight data hierarchical storage management program is stored, wherein when the multidimensional flight data hierarchical storage management program is executed by a processor, the steps of the multidimensional flight data hierarchical storage management method provided in the first aspect are implemented.

[0019] The multi-dimensional flight data hierarchical storage management method, system, device, and storage medium provided by this invention have the following significant beneficial effects: By coordinating the "status monitoring and triggering" and "degradation data filtering" steps, this method effectively overcomes the problems of erroneous degradation and scenario mismatch caused by "rigid decision-making" in traditional solutions. Instead of mechanically triggering based solely on a single capacity threshold, this method monitors multi-dimensional states in real time and filters data based on multi-dimensional attributes and classification labels, ensuring that degradation decisions can dynamically adapt to the rapid transitions in UAV mission phases. This avoids mistakenly degrading high-value or soon-to-be-accessed "hot data" under storage pressure, thus guaranteeing real-time data access performance during critical mission execution and achieving a fundamental shift from "one-size-fits-all" to "precise identification."

[0020] By employing a "dynamic path planning" step, the inefficient resource release problem caused by the traditional "step-by-step degradation" path is completely resolved. This method does not follow a fixed path, but rather dynamically plans the optimal degradation path for each target data based on compliance rules and intelligent models. This enables the system to intelligently select the optimal cross-level path, such as "L1→L3," when faced with urgent space release needs, significantly shortening the migration chain and freeing up valuable high-speed storage space for subsequent tasks with minimal cost and maximum speed. This significantly improves the recycling efficiency of storage resources and the system's agility in responding to sudden loads.

[0021] By integrating compliance screening and intelligent decision-making in "dynamic path planning," this method successfully addresses the complex optimization challenges of dynamic trade-offs among multiple objectives. It uses mandatory compliance constraints as initial screening criteria, followed by comprehensive optimization using a degradation decision model. This intelligently balances multiple objectives such as space release, performance impact, and migration costs while strictly meeting data security and regulatory requirements. This enables adaptive adjustment and Pareto optimization of storage strategies under different UAV mission modes (such as performance-first mode and cost-saving mode), reducing reliance on manual parameter tuning and enhancing the intelligence level of system management.

[0022] The overall technical solution effectively avoids system stability risks caused by passive response mechanisms through the orderly connection and closed-loop execution of the above steps. The complete process, from precise triggering and screening to optimized planning and controllable migration, transforms data degradation operations from passive, centralized "avalanche" processing into predictable and smoothly executed routine scheduling of system resources. This significantly reduces I / O interference and performance jitter caused by storage management operations to real-time tasks, ensuring high reliability and service continuity of the UAV data storage subsystem during long-term, complex task execution. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic flowchart of a method according to an embodiment of the present invention.

[0025] Figure 2 This is a schematic diagram of a hierarchical storage system according to an embodiment of the present invention.

[0026] Figure 3 This is a schematic diagram of the decision-downgrade migration path process of a method according to an embodiment of the present invention.

[0027] Figure 4 This is a schematic block diagram of a system according to an embodiment of the present invention.

[0028] Figure 5 This is a schematic diagram of the structure of a device provided in an embodiment of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0031] The multidimensional flight data hierarchical storage and management method provided in this embodiment of the invention is executed by a computer device, and correspondingly, the multidimensional flight data hierarchical storage and management system runs on the computer device.

[0032] Figure 1 This is a schematic flowchart illustrating a method according to an embodiment of the present invention. Wherein, Figure 1 The implementing entity can be a multi-dimensional flight data hierarchical storage and management system. Depending on different needs, the order of the steps in this flowchart can be changed, and some can be omitted.

[0033] like Figure 1 As shown, the method includes: S1. Monitor the resource status of the pre-built hierarchical storage system in real time; when the status of any storage node meets the preset degradation trigger condition, the storage node is identified as the source node to be degraded; S2. Based on the access frequency, data value, and data classification tags of the data shards in the source node to be downgraded, select the target data to be downgraded; the data classification tags are at least associated with data sensitivity level and storage compliance rules; S3. Plan a degradation migration path for the target degraded data. The planning process includes: excluding all non-compliant potential degradation paths according to the storage compliance rules associated with the target degraded data to obtain a candidate path set; and using a degradation decision model, determining the optimal degradation path from the candidate path set based on the real-time status of the source node to be degraded and the downstream target node, the attributes of the target degraded data, and the migration cost-benefit model. S4. According to the optimal degradation path, migrate the target degradation data to the downstream storage node and update the data access route.

[0034] In one embodiment of the present invention, based on step S1, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0035] Figure 2 A schematic diagram of a four-level hierarchical storage system provided in an embodiment of the present invention is shown. The system consists of four levels with different physical performance and cost characteristics, forming a continuous storage hierarchy from a "high-speed hot layer" to a "high-capacity cold layer".

[0036] Primary storage node: High-speed access layer Storage media: Solid-state drives (SSDs) or non-volatile memory (NVM) based on NVMe or SATA protocols are used, and in-memory databases (such as Redis) can be integrated as hot data caches.

[0037] This layer offers the highest data access performance (high IOPS, low latency), but also has the highest unit storage cost. Its primary design goal is to ensure the real-time performance of tasks.

[0038] Mapped multidimensional flight data types include: key intermediate data in real-time processing, such as intermediate feature vectors of target recognition algorithms and current frame buffer data in real-time stitching; high-priority metadata and indexes, such as flight trajectories generated during critical mission periods, sensor status, and active portions of mission event logs; frequently accessed control and status data, such as real-time status information of the flight control system and the current mission instruction set; and small batches of high-value sample data that are being actively analyzed.

[0039] Secondary storage nodes: performance and capacity balancing layer Storage media: A storage array that uses a hybrid of solid-state storage (such as SATA SSD) and hard disk drives (HDD), which can optimize access to hot data through automatic tiering technology (such as SSD caching).

[0040] It strikes a balance between performance and capacity, providing random access capabilities superior to pure mechanical storage, with a significantly lower unit cost than Tier 1 storage. Its design goal is to support high-frequency access and fast analysis.

[0041] Mapped multidimensional flight data types: Structured data that has completed initial processing: Metadata files with completed target annotations, and structured task data integrated by Geographic Information System (GIS); Complete datasets of recent tasks: Processed imagery, point clouds, and associated engineering files of tasks executed in the past few hours to days; Models and configuration files: Airborne AI inference models, frequently called task planning templates; System and application logs: System logs used for recent fault diagnosis and performance analysis.

[0042] Tier 3 storage nodes: High-capacity access layer Storage media: RAID arrays or storage servers composed of high-density mechanical hard drives (such as large-capacity enterprise-grade HDDs).

[0043] Its core advantages are high storage capacity and low cost per TB, making it suitable for sequential large-block data transfer, but its random access performance is relatively low. Its design goal is to economically store massive amounts of raw data and archived copies.

[0044] Mapped multidimensional flight data types: Raw sensor data: Raw high-definition / hyperspectral video and raw lidar point cloud data collected in one or more missions; Long-term project data: Centralized backup of raw data and processing intermediates of all missions throughout the entire project cycle; Rare historical data: Historical mission data with extremely low access frequency but not yet reaching the archiving period; Batch processing result data: Large files such as large-area orthophoto maps and 3D reconstruction models.

[0045] Level 4 storage node: Long-term archive layer Storage media: sequential access media such as linear magnetic tape open technology (LTO) tape libraries, high-capacity SATA hard disk archive arrays, or Blu-ray disc libraries.

[0046] It offers the lowest unit storage cost and extremely long data retention life (typically over 10 years), but with the highest data retrieval latency (minutes to hours). Its design aims to meet regulatory compliance and long-term retention of historical data.

[0047] Mapped multidimensional flight data types: Long-term archive data required by regulations: Original data and final results that must be preserved for several years or even decades according to industry regulations (such as surveying, environmental protection, and national defense); Permanent historical archives: Complete datasets of landmark missions with important historical or research value; Disaster recovery backups: Offline cold backup copies of core data; Data from terminated projects that have passed their active period: The final destination of all relevant data after the project is completely completed.

[0048] In this embodiment, the lifecycle of typical UAV flight data (such as a reconnaissance video) is as follows: After acquisition, it is first stored in Level 1 storage for real-time analysis; after the real-time mission ends, the processed metadata and key segments may be retained in Level 1 or downgraded to Level 2 storage for mission review and quick retrieval; the original video file may be migrated to Level 3 storage in the short term; according to the data retention policy, the video and its metadata are finally migrated to Level 4 storage for long-term archiving. This flow process is not fixed, but is dynamically and intelligently scheduled by the adaptive degradation management method described in this invention based on the real-time data access mode, value weight, and system resource status.

[0049] S101. Through lightweight agents deployed on storage nodes at all levels and central monitoring services, the status of the following key resources is periodically collected and aggregated at the millisecond to second level: Capacity status is monitored periodically by calling operating system or storage device management interfaces (such as df, lsblk, or the storage array's RESTful API), including real-time monitoring of the logical capacity utilization and physical storage block utilization of each storage node. Logical capacity utilization reflects the space occupancy at the file system level, while physical storage block utilization more accurately reflects the actual consumption of the underlying storage media, which is especially important for media with write amplification effects, such as flash memory.

[0050] Raw data is obtained through performance monitoring tools (such as iostat, sar) or performance counters built into storage devices, and then aggregated and normalized by the monitoring service to obtain performance metrics for monitoring, including: Input / output operations per second (IOPS): This measures the number of read / write operations completed per second, distinguishing between random and sequential read / write operations. This is a key metric for measuring a storage node's ability to handle concurrent requests, especially crucial for Tier 1 and Tier 2 storage nodes.

[0051] Throughput: Monitors the amount of data transferred per second (MB / s or GB / s). This metric reflects the storage node's ability to process large blocks of continuous data and is particularly important for evaluating the data transfer efficiency of Tier 3 and Tier 4 storage nodes.

[0052] Access Latency: Monitors the average time, 95th percentile (P95), and 99th percentile (P99) latency required from receiving an I / O request to completing it. Low latency is crucial for ensuring the performance of real-time UAV missions.

[0053] Other performance metrics: Depending on the characteristics of the storage medium, supplementary monitoring metrics such as cache hit rate (for secondary hybrid storage), queue depth, and error rate should be performed.

[0054] Health status monitoring: Media health: For flash memory media, monitor the remaining spare block ratio, wear leveling count, and uncorrectable error rate; for hard disk drives, monitor SMART attributes (such as reallocation sector count and seek error rate).

[0055] System load: Monitor the CPU utilization, memory utilization, and network bandwidth usage of the server where the storage node is located to determine whether the performance bottleneck is caused by the storage subsystem itself.

[0056] S102. Set downgrade trigger conditions The degradation triggering conditions consist of a set of configurable thresholds and rules, which are divided into two categories: capacity triggering conditions and performance triggering conditions. The two are in an "OR" logical relationship, and the degradation decision process is triggered when either condition is met.

[0057] Capacity triggering conditions: Threshold setting: A dynamic or static capacity utilization threshold (T_capacity) is preset for each level of storage node. For example, in one embodiment, it can be set as follows: Level 1 storage node: T_capacity = 85% (early warning), 90% (critical), 95% (urgent).

[0058] Secondary storage node: T_capacity=88% (warning), 93% (critical), 98% (urgent).

[0059] Level 3 / 4 storage nodes: The threshold can be relaxed appropriately, but an absolute threshold (such as 98%) must be set to prevent them from being completely filled.

[0060] Triggering logic: When the current capacity utilization rate of a storage node is continuously exceeded by its preset threshold for a certain period of time (such as 3 consecutive sampling periods), the capacity triggering condition is determined to be met.

[0061] Performance trigger conditions: Baseline establishment: A dynamic baseline is established for each storage node's key performance indicators (IOPS, throughput, P95 latency). The baseline is not a fixed value, but rather an expected performance level calculated using statistical methods (such as moving averages) based on historical data (such as performance under similar loads over the past 24 hours).

[0062] Performance degradation determination: Set a threshold for the magnitude of performance degradation (T_degradation, e.g., 15%, 25%, 40%). When the current performance metric (e.g., IOPS) is detected to be lower than its dynamic baseline value (100% - T_degradation), performance degradation is determined to have occurred. For example, if the baseline IOPS of a Tier 1 storage node is 10,000 and T_degradation is set to 25%, then a performance degradation alarm will be triggered when the real-time IOPS remains below 7,500.

[0063] Correlation analysis: Analyze whether the performance degradation is caused by near-saturation of capacity, decline in media health, or abnormal surge in front-end load, in order to help determine the most fundamental triggering cause.

[0064] S103. Identify the source node to be downgraded. Once a storage node is determined to meet any of the above degradation triggering conditions, that node is marked as a "source node to be degraded". The determination process includes the following steps: Event Generation: The monitoring service generates a structured "degradation trigger event", which includes: node identifier (e.g., Storage_Node_L1_01); trigger type (capacity trigger or performance trigger); specific metric values ​​at the time of triggering (e.g., utilization = 92%, IOPS decay = 30%); timestamp and event severity level (warning, critical, emergency).

[0065] Node Status Snapshot: Immediately captures a complete status snapshot of the source node to be degraded, including its current capacity details, real-time performance metric values, health status, and metadata summaries of all data shards mounted on it. This snapshot will serve as an important input for subsequent data filtering and path planning.

[0066] The event was pushed to the degradation decision engine, officially initiating the subsequent "degradation data filtering" and "dynamic path planning" processes.

[0067] In one embodiment of the present invention, based on step S2, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0068] This step abandons simple, single-dimensional strategies such as "Least Recently Used (LRU)" and instead uses a comprehensive evaluation model with quantifiable, configurable, and adaptive weights to accurately identify the most suitable "target degradation data" for migration from the source nodes to be downgraded.

[0069] S201. Construction and Input of Data Evaluation Model The data evaluation model aims to calculate a "degradation suitability score" (Score_degrade) for each data shard (such as a file, object, or data block) in the source node to be downgraded. A higher score indicates that the data shard is more suitable for downgrading. The model's input comes from three dimensions of data attributes: (1) Access frequency dimension: Data Acquisition: The system continuously records the number of accesses (including reads and writes) for each data shard within the most recent time window (e.g., the past 24 hours, 7 days). Access Frequency (F) access It can be normalized to the interval [0,1], where 1 indicates the most frequent access.

[0070] Rating Calculation: Visit Frequency Rating (S) freq ) and F access Negative correlation. One implementation is: S freq =1-F access This means that data that has not been accessed for a long time scores highly in this area.

[0071] (2) Value weight dimension: Weight calculation basis: Assign a value weight (W) to each data shard. value This weight is a combination of one or more of the following factors: Business importance: Marked by the task planning system or operator. For example, data involving critical objectives has a weight of 0.9, while general environmental monitoring data has a weight of 0.3.

[0072] Generation / Reconstruction Cost: Estimates the resources (time, energy consumption, computation cycle) required to re-collect or regenerate the data. The higher the cost, the greater the weight.

[0073] Business relevance: The degree to which data is closely related to other active tasks, current analytical models, or core systems.

[0074] Rating Calculation: Value Weighted Rating (S) value ) and W value Negative correlation. One implementation is: S value =1-W value The lower the value of the data, the more suitable it is to be downgraded.

[0075] Compliance Dimension (based on data classification tags): Tagging system: Each data slice is assigned a set of structured data classification labels upon creation or ingestion, which includes at least: Sensitivity levels: such as "public", "internal", "secret", "top secret".

[0076] Retention period: such as "short-term (30 days)", "medium-term (1 year)", "long-term (10 years)" or "permanent".

[0077] Other compliance requirements include: "geofencing restrictions", "audit trail requirements", and "encryption status".

[0078] Scoring Calculation: Compliance Score (S) compliance This is a rule-based mapping score that reflects the "compliance difficulty" or "urgency" of data downgrade. For example: Data labeled "public" and whose "short-term" retention period has expired, S compliance It can be set to 1.0 (downgrading is very compliant and urgent).

[0079] Data tagged "secret" and whose retention period of "long term" has not expired, S compliance It can be set to 0.1 (downgrading should be done with extreme caution, and downgrading to a lower security level may not be allowed).

[0080] This rating ensures that downgrade operations are always conducted within a pre-defined compliance framework.

[0081] S202. Comprehensive Score Calculation and Dynamic Weight Adjustment The overall degradability score for data sharding is calculated using the following formula: Score degrade =W1×S freq +W2×S value +W3×S compliance The weighting coefficients W1, W2, and W3 are not fixed, but are dynamically adjusted according to the current system warning status of the "source node to be downgraded" to achieve policy bias in different scenarios: Normal / Warning Status: The system has ample buffer time. Weighting is biased towards slightly higher values ​​for W2 (value weight) and W3 (compliance), emphasizing the orderly migration of cold data and optimization of storage structure while ensuring data value and absolute compliance. For example: (W1, W2, W3) = (0.4, 0.3, 0.3).

[0082] Critical / Emergency State: The system needs to quickly free up space or alleviate performance pressure. Weighting will heavily favor W1 (access frequency), prioritizing the migration of the "coldest" data to free up space as quickly as possible, while maintaining minimum compliance requirements (W3). For example: (W1, W2, W3) = (0.7, 0.1, 0.2). In extreme emergency states, W1 may be even higher.

[0083] S203. Filtering and Output of Target Degradation Data Batch scoring: Once the source node to be downgraded is determined, the system calls the model to perform fast batch scoring on all candidate data shards on it.

[0084] Threshold filtering: Set a dynamic or static scoring threshold (T) score All Scores degrade >T score Data shards will be initially selected as candidates.

[0085] Strategy sorting and selection: Based on the current strategy (such as "release maximum space" or "migrate the fewest files"), the candidate list is sorted, and the top-ranked data shards are selected to finally determine the "target downgrade data" set for this downgrade operation.

[0086] Output: The output results not only include the target data list, but also the score details of each data segment and the selected degradation path (determined by subsequent modules), providing a complete basis for execution and auditing.

[0087] In one example, after a drone swarm mapping mission, a secondary storage node (hybrid storage) reached 88% capacity utilization due to caching a large amount of intermediate processing data, triggering an alert. The system designated it as a source node to be downgraded and placed it in an "alert state," with weights set to (0.4, 0.3, 0.3). The model then scores the 100,000 data shards on this node.

[0088] An orthophoto map of an old project (value weight 0.2, labeled "internal / long-term") generated 3 months ago and inaccessible for nearly 1 month, with its S... freq High, S value High, S compliance Medium, Score degrade The calculated value is 0.82.

[0089] A high-resolution 3D model (value weight 0.9, tagged "internal / mid-term") that was just processed yesterday but has already been marked as a "key achievement" is currently unaccessed (S). freq High), but because its value weight is extremely high (S value Extremely low), Score degrade Only 0.35. Set T score =0.6, the old imagery was successfully filtered out as target downgrade data, while the critical 3D model was preserved. This process ensured that core business data was protected while alleviating storage pressure.

[0090] In one embodiment of the present invention, based on step S3, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0091] S301. Based on the storage compliance rules associated with the target degraded data, exclude all non-compliant potential degrade paths to obtain a candidate path set, including: S301.1 Definition and Mapping of Storage Compliance Rules A pre-defined storage compliance rule base is provided, where each rule defines mandatory requirements for data storage, and these rules are logically bound to data classification tags.

[0092] Example of compliance rules: Rule R1 (based on sensitivity level): "Data with a sensitivity level of 'secret' or higher must be stored on storage nodes that support hardware encryption and have a physical security level of 'Tier-3' or higher." Rule R2 (based on retention period and media): "For data with a retention period of 'long-term' or 'permanent', the final archived copy must be stored on WORM (write-once, read-many) media or on storage nodes with tamper-proof auditing capabilities." Rule R3 (based on geographic / jurisdictional boundaries): "Data containing personal information from a specific geographic region must be stored at a location whose physical location does not exceed the specified geofence." Rule R4 (based on data type): "Raw flight logs and sensor calibration data must not be removed from primary or secondary storage nodes for 6 months after the mission ends to ensure rapid fault diagnosis." Tag-to-rule mapping: When a piece of data is tagged as {Sensitivity Level: Secret, Retention Period: Long-term, Data Type: Flight Log}, the system will automatically associate it with a set of rules {R1, R2, R4}.

[0093] This mapping is completed during the data governance strategy configuration phase to ensure consistency between labels and compliance requirements.

[0094] S301.2 Enumeration of Potential Migration Paths Given the selected "target data to be downgraded" and "source node to be downgraded" (assuming it is L2), the system will automatically enumerate all theoretically feasible downgrade paths: Direct migration path: Migrate directly from the source node to any of its downstream nodes. For example: L2 → L3, L2 → L4.

[0095] Step-by-step migration path: Migrate in stages, passing through intermediate nodes. For example: L2 → L3 → L4.

[0096] The enumeration range allows migration from a higher level to any lower level. All enumerated paths constitute the "set of potential migration paths".

[0097] S301.3 Path Compliance Verification Each potential migration path is iterated through one by one, and the attributes of its final target storage node (the last level node for a staircase path) and optional relay nodes are verified.

[0098] Verification process: For each mandatory compliance rule (such as R1, R2, R4) associated with the target downgraded data, perform the following logical judgment: The storage node attribute requirements specified in the read rules (such as "supports hardware encryption", "physical security level ≥ Tier-3", "WORM function").

[0099] Retrieves a list of attributes for the target nodes (and relay nodes, if applicable) along the path. This list is defined and verified during storage node registration and includes information such as technical specifications, security certifications, and geographical location.

[0100] Perform a match: Determine if the actual attributes of the node fully meet the rule requirements. A match is a boolean value (yes / no).

[0101] Verification logic: A path is considered compliant only if it meets all associated compliance rules; if any rule is not met, the path is immediately excluded.

[0102] The verification process is asymmetric: the rules only constrain the attributes that the target node must meet, but do not prohibit data from migrating from high-security / high-performance nodes to eligible lower-level nodes.

[0103] S301.4 Generation of Candidate Path Set After traversing and verifying all potential paths, all paths that pass the verification are included in the "compliant candidate path set".

[0104] If the set is empty after verification, it means that under the current architecture, there is no legitimate degradation path for the target data. An advanced alert will be triggered, notifying the administrator that data labels need to be adjusted, compliance rules modified, or storage node capabilities expanded, without performing any illegal operations.

[0105] S302. Using a degradation decision model, based on the real-time status of the source node to be downgraded and the downstream target node, the attributes of the target degradation data, and a migration cost-benefit model, determine the optimal degradation path from the candidate path set. Please refer to [reference needed]. Figure 3 ,include: Figure 3 This diagram illustrates a path optimization decision-making process based on deep reinforcement learning, as provided in an embodiment of the present invention.

[0106] S302.1 Construction of the State Input Vector For each candidate path that passes the compliance screening, construct a high-dimensional state input vector in a unified format. Vector This vector fully encapsulates all the contextual information needed for decision-making. Its structure contains at least the following four parts: Node state context: Source node status: Real-time capacity utilization, current IOPS, throughput, average latency, and health score of the source node to be degraded.

[0107] Target node status: Real-time capacity utilization of the final target node of the candidate path, expected utilization after receiving data, current load (IOPS, throughput utilization), available bandwidth, and remaining lifetime of storage media.

[0108] Data attribute context: Target downgrade data attributes: data shard size, value weight score calculated by the data evaluation model (S value ), visit frequency score (S) freq ), compliance score (S) compliance ), and the vectorized representation of data classification labels.

[0109] Preliminary cost-benefit assessment of the route: Estimated migration costs: Quickly calculated using a lightweight migration cost-benefit model, including: Time cost: The estimated migration time based on data size, available bandwidth between source and target nodes, and current load.

[0110] Resource overhead: The estimated CPU, memory, and network I / O resources required during the migration process.

[0111] Performance impact assessment: A quantitative assessment of the impact of the migration operation on the front-end business performance of the source and target nodes (such as the expected percentage increase in latency).

[0112] Space release benefit: The logical storage space expected to be released on the source node during this migration.

[0113] System and environment context: System global load: The average load level of the entire storage cluster.

[0114] Mission phase information: The current mission phase of the drone (such as "reconnaissance", "returning home", "data analysis"). This information can indicate the sensitive period for system performance.

[0115] S302.2 Degradation Decision Model: Deep Reinforcement Learning Policy Network This implementation uses deep reinforcement learning (DRL), specifically value-based deep Q-networks (DQN) or variants such as Dueling DQN, as the core of the degradation decision model.

[0116] Model architecture: The policy network is a deep neural network whose input layer dimension is the same as the state input vector. Vector The dimensions match.

[0117] The network contains multiple fully connected hidden layers for learning high-order representations of state features.

[0118] Each neuron in the output layer corresponds to a candidate path, and its output value (Q-value) represents the expected long-term cumulative reward that can be obtained by choosing that path given the current state.

[0119] Training process and long-term reward design: Training environment: Training is performed using historical or synthetic storage load data in a simulated or production environment shadow mode.

[0120] State (S): that is, the State constructed above. Vector .

[0121] Action (A): The degraded path chosen by the agent (i.e., the output layer neuron index).

[0122] Reward (R): After each downgrade operation is executed and the network runs stably for a period of time (e.g., 1 hour), the system calculates a long-term reward to guide network learning. The reward function R is a combination of multiple metrics: R space Space release reward. Proportional to the effective space released, but a penalty is imposed when the released space does not reach the expected level.

[0123] R performance Performance-based rewards: If the key performance indicators (such as P95 latency) of the source and target nodes do not deteriorate or even improve after migration, a positive reward will be given; if it causes a significant performance decline, a negative reward will be given.

[0124] R compliance Compliance Rewards: A basic positive reward is given for consistently meeting compliance requirements; additional rewards are given if clever path selection reduces management complexity while meeting compliance requirements (e.g., avoiding the use of temporary relay nodes).

[0125] R efficiency Efficiency rewards. Rewards actions that complete migrations quickly and with low resource consumption.

[0126] Total reward: R = α × R space +β×R performance +γ×R compliance +δ×R efficiency The weights α, β, γ, and δ represent the system's preference for different objectives.

[0127] Training objective: The training objective of the policy network is to maximize the expected value of the cumulative reward for future discounts. , where γ is a discount factor, enabling the model to have long-term planning capabilities.

[0128] S302.3 Online Reasoning and Path Scoring During the online decision-making phase, given the "target degradation data" and the "compliance candidate path set": State vector construction and model input: Construct the corresponding State for each candidate path. Vector They are then input in batches into the pre-trained policy network.

[0129] Overall Score Generation: The policy network outputs a Q-score for each candidate path. This Q-score is the overall score for that path, essentially a prediction of long-term future returns based on historical experience. A higher score indicates that choosing that path is more beneficial to the system in the long run.

[0130] Post-processing of scores: In some implementations, the Q-value can be weighted and combined with other deterministic business rules (such as prioritizing the performance of a critical application) to generate the final score for ranking.

[0131] S302.4 Selection of the Optimal Degradation Path The candidate path with the highest overall score is selected as the "optimal degradation path" for this degradation operation. This decision is made instantaneously (usually <100ms) and incorporates complex multi-objective trade-offs and long-term benefit considerations.

[0132] In one embodiment of the present invention, based on step S4, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0133] Phase 1: Pre-migration Preparation This stage logically "locks in" the target data and establishes a migration framework. All operations are completed instantaneously at the metadata level without affecting the data itself.

[0134] Data Tagging and State Locking: In the global metadata service, the status field of the unique identifier (such as Inode number or object ID) of the selected "target degradation data" is updated to "Pending Migration". This status tag prevents other management processes (such as deduplication and compression) from concurrently modifying this data, ensuring that the data content remains static during migration.

[0135] Create Redirect Information: Create a redirect entry for this data in the metadata. This entry is a temporary, versioned record containing the following key information: Data Identifier; Source Location: The physical storage node and path where the data is currently located; Target Location: The final downstream storage node and pre-assigned path determined by the "Optimal Degradation Path"; Migration Status: Initialized to "Preparing"; Checksum Information: Calculate the cryptographic hash (such as SHA-256) of the source data and record it here for subsequent integrity verification.

[0136] Update routing logic: The unified data access gateway (or the client for each storage node) is configured such that when it receives an access request for data in the "pending migration" state, it first queries the redirection entry and, based on the state and policy in the entry, determines whether the request should be routed to the source or destination location. At this point, the routing logic is initialized to "read-oriented source, write-deferred or synchronous dual-write".

[0137] Phase Two: Asynchronous Migration Execution During this phase, data blocks are physically moved in the background, while the foreground access is transparently served through a redirection mechanism.

[0138] Background data replication: A dedicated migration service process is started, responsible for asynchronously replicating data from the source storage node to the target storage node. The migration process supports breakpoint resumption and traffic control. The traffic control strategy is dynamically adjusted based on the real-time system load; for example, during peak business periods, it automatically reduces migration bandwidth usage to avoid impacting foreground I / O performance.

[0139] Transparent access support: Read requests: Before the data is fully migrated, all read requests are redirected to the source location to ensure that the client receives the latest data; Write requests: Different strategies are adopted based on the configured data consistency level: Strong consistency mode: Write requests are paused until the migration is complete, or synchronous dual-write (writing to both the source and target simultaneously) is used to ensure consistency between the two ends. This mode is suitable for critical metadata. Eventual consistency mode: Write requests are logged to the incremental log associated with the redirection entry. After the data block migration is complete, these incremental logs are applied to the target location. This mode is suitable for most business data and has minimal impact on business operations.

[0140] Progress monitoring and anomaly handling: The migration service updates the "migration status" and "progress percentage" in the redirection entry in real time. If an anomaly such as network interruption or node failure occurs during the migration process, the migration job will be paused and a checkpoint will be recorded. After the fault is recovered, the migration can continue from the checkpoint without restarting.

[0141] Phase 3: Switching, Verification, and Cleanup This stage involves switching data identities and reclaiming resources.

[0142] Data integrity verification: After the background migration service reports that data replication is complete, it does not switch immediately. It triggers an integrity verification job. This job reads the newly written data at the target location, recalculates its checksum, and compares it with the checksum of the source data recorded during the preparation phase. The verification passes only if the two are completely identical.

[0143] Atomic route switching: After successful verification, perform an atomic metadata update operation: update the data's status from "Pending Migration" to "Normal". Update the data's permanent physical location information to the target location. Delete or invalidate temporary redirect entries.

[0144] This atomic operation ensures that, at a certain point in time, the access route view for the same data for all clients switches instantaneously and consistently, preventing a chaotic situation where some requests go to the source and others to the destination.

[0145] Source space reclamation: After a successful routing switch, the original data blocks on the source storage node become orphan data. An instruction is sent to the source node to mark them as reclaimable space. Actual physical space reclamation (such as the TRIM command on an SSD, or a garbage collection mechanism) is performed asynchronously by the storage node in the background at an appropriate time, avoiding instantaneous I / O pressure.

[0146] Completion notification and audit: Generate a complete "downgrade operation completed" audit log, recording data identifier, path, time consumed, amount of space released, verification results, etc., and notify the monitoring system.

[0147] In some embodiments, the multidimensional flight data hierarchical storage and management system may include multiple functional modules composed of computer program segments. The computer programs of each program segment in the multidimensional flight data hierarchical storage and management system may be stored in the memory of a computer device and executed by at least one processor to perform (see details). Figure 1 (Description) Functionality of hierarchical storage and management of multidimensional flight data.

[0148] In this embodiment, the multi-dimensional flight data hierarchical storage and management system can be divided into multiple functional modules according to its functions, such as... Figure 4 As shown. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, and is stored in memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0149] The real-time monitoring module is used to monitor the resource status of the pre-built hierarchical storage system in real time; when the status of any storage node meets the preset degradation trigger condition, the storage node is identified as the source node to be degraded. The target determination module is used to filter out target degraded data based on the access frequency, data value, and data classification tags of the data shards in the source node to be degraded; the data classification tags are at least associated with data sensitivity level and storage compliance rules; The path planning module is used to plan a degradation migration path for the target degradation data. The planning process includes: excluding all non-compliant potential degradation paths according to the storage compliance rules associated with the target degradation data to obtain a candidate path set; and using a degradation decision model, determining the optimal degradation path from the candidate path set based on the real-time status of the source node to be downgraded and the downstream target node, the attributes of the target degradation data, and the migration cost-benefit model. The degradation execution module is used to migrate the target degradation data to the downstream storage node according to the optimal degradation path and update the data access route.

[0150] Figure 5 The multi-dimensional flight data hierarchical storage management method provided in the embodiments of this application can be applied to devices. Those skilled in the art will understand that the device structure involved in the embodiments of this invention does not constitute a limitation on the device. A device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the device includes, but is not limited to, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0151] The device 500 may include a processor 510, a memory 520, and a communication unit 530. These components communicate via one or more buses. Those skilled in the art will understand that the server structure shown in the figure does not constitute a limitation of the present invention. It may be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0152] The memory 520 can be used to store execution instructions of the processor 510. The memory 520 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. When the execution instructions in the memory 520 are executed by the processor 510, the device 500 is able to perform some or all of the steps in the above method embodiments.

[0153] The processor 510 serves as the control center of the storage device, connecting various parts of the electronic device via various interfaces and lines. It executes software programs and / or modules stored in the memory 520, and calls data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 510 may consist only of a central processing unit (CPU). In this embodiment of the invention, the CPU may have a single processing core or include multiple processing cores.

[0154] The communication unit 530 is used to establish a communication channel, enabling the storage device to communicate with other devices. It can receive user data sent by other devices or send user data to other devices.

[0155] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, which, when executed, may include some or all of the steps provided in the embodiments of the present invention. The storage medium may be a magnetic disk, an optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0156] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or any other medium capable of storing program code. It includes several instructions to cause a computer device (which may be a personal computer, a server, or a second device, network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0157] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0158] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or modules may be electrical, mechanical, or other forms.

[0159] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0160] In addition, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0161] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the present invention by those skilled in the art without departing from the spirit and essence of the invention, and such modifications or substitutions should all be within the scope of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should also be covered within the protection scope of the present invention.

Claims

1. A hierarchical storage and management method for multidimensional flight data, characterized in that, include: Real-time monitoring of the resource status of the pre-built tiered storage system; When the state of any storage node meets the preset degradation trigger condition, the storage node is determined as the source node to be degraded. Based on the access frequency, data value, and data classification tags of the data shards in the source node to be downgraded, the target downgraded data is selected. The data classification labels are at least associated with data sensitivity levels and storage compliance rules; The process of planning a degradation migration path for the target degradation data includes: eliminating all non-compliant potential degradation paths based on the storage compliance rules associated with the target degradation data to obtain a candidate path set; and using a degradation decision model, determining the optimal degradation path from the candidate path set based on the real-time status of the source node to be downgraded and the downstream target node, the attributes of the target degradation data, and a migration cost-benefit model. According to the optimal degradation path, the target degradation data is migrated to the downstream storage node, and the data access route is updated.

2. The method according to claim 1, characterized in that, The hierarchical storage system includes: As a primary storage node in the high-speed access layer, it uses flash-based storage media; As a secondary storage node that balances performance and capacity, it uses a hybrid storage medium of solid-state storage and mechanical storage. As a tertiary storage node serving as a high-capacity access layer, it employs high-density mechanical storage media; As a fourth-level storage node for long-term archiving, a tape library or high-capacity sequential access storage medium is used. Among them, the data access performance of storage nodes decreases sequentially from level 1 to level 4, and the unit storage cost decreases sequentially.

3. The method according to claim 1, characterized in that, The downgrade triggering conditions include: The current capacity utilization rate of the storage node exceeds the preset capacity threshold; The current performance metrics of the storage node have degraded by more than a preset amount compared to the baseline value; The performance metrics include at least one of input / output operation count, throughput, or access latency.

4. The method according to claim 1, characterized in that, Based on the access frequency, data value, and data classification tags of the data shards in the source node to be downgraded, the target data for downgrading is selected, including: Based on a pre-built data evaluation model, each data shard in the source node to be downgraded is scored. The input of the data evaluation model includes at least: the historical access frequency of the data shard; the calculated value weight of the data shard, which is determined based on at least one of the importance of the data, the generation cost, or the business relevance; and the data classification label associated with the data shard. Select data segments with scores higher than a preset threshold as the target downgrade data; The data evaluation model includes: Overall score = W1 × visit frequency score + W2 × value weight score + W3 × compliance score; The access frequency score is negatively correlated with the historical access frequency of the data shard; the value weight score is calculated based on the importance, generation cost, or business relevance of the data shard; the compliance score is determined based on the data sensitivity level and retention period requirements indicated by the data classification label; and the weight coefficients W1, W2, and W3 are dynamically adjusted according to the system warning status of the source node to be downgraded.

5. The method according to claim 1, characterized in that, Based on the storage compliance rules associated with the target degraded data, all non-compliant potential degrade paths are excluded, resulting in a candidate path set, including: Based on the data classification tags associated with the target downgraded data, obtain the corresponding set of mandatory storage compliance rules; Traverse all potential migration paths from the source node to be degraded to its downstream storage nodes at each level. Potential migration paths include direct migration paths or stepped migration paths. For each of the potential migration paths, verify whether the attributes of its target storage node satisfy each storage compliance rule in the storage compliance rule set; Potential migration paths that meet all storage compliance rules are included in the candidate path set.

6. The method according to claim 1, characterized in that, Using a degradation decision model, based on the real-time status of the source node to be downgraded and the downstream target node, the attributes of the target degradation data, and a migration cost-benefit model, the optimal degradation path is determined from the candidate path set, including: Construct a state input vector for path evaluation, the state input vector including at least: the real-time performance and capacity status of the source node to be degraded and the downstream target node, the multi-dimensional attributes of the target degradation data, and the pre-evaluation results of the migration cost-benefit model for the candidate path; The state input vector is input into a pre-trained degradation decision model; the degradation decision model is a policy network trained based on deep reinforcement learning; the policy network is trained to maximize long-term rewards, and the long-term rewards are calculated based on the system space state after historical degradation operations, changes in performance indicators, and compliance audit results. The degradation decision model outputs a comprehensive score for each candidate path. The comprehensive score is obtained by weighted calculation of migration cost, space release benefit, performance impact assessment and compliance satisfaction. The candidate path with the highest comprehensive score is selected as the optimal degradation path.

7. The method according to claim 1, characterized in that, According to the optimal degradation path, the target degradation data is migrated to the downstream storage node, and the data access route is updated, including: During the preparation phase, the target degraded data is marked as pending migration, and redirection information pointing to the target downstream storage node is pre-registered in the metadata. During the execution phase, the data content is asynchronously migrated to the downstream storage node determined by the optimal degradation path in the background. During the migration, access requests for the data are transparently redirected to the original or target location through the redirection information. In the completion phase, after verifying the integrity of the target location data, the redirection information in the metadata is updated to permanent routing information, and the original data space in the source storage node is reclaimed.

8. A multi-dimensional flight data hierarchical storage and management system, characterized in that, include: The real-time monitoring module is used to monitor the resource status of the pre-built hierarchical storage system in real time. When the state of any storage node meets the preset degradation trigger condition, the storage node is determined as the source node to be degraded. The target determination module is used to filter out target degradation data based on the access frequency, data value, and data classification tags of the data shards in the source node to be downgraded. The data classification labels are at least associated with data sensitivity levels and storage compliance rules; The path planning module is used to plan a degradation migration path for the target degradation data. The planning process includes: excluding all non-compliant potential degradation paths according to the storage compliance rules associated with the target degradation data to obtain a candidate path set; and using a degradation decision model, determining the optimal degradation path from the candidate path set based on the real-time status of the source node to be downgraded and the downstream target node, the attributes of the target degradation data, and the migration cost-benefit model. The degradation execution module is used to migrate the target degradation data to the downstream storage node according to the optimal degradation path and update the data access route.

9. A multi-dimensional flight data hierarchical storage and management device, characterized in that, include: The memory is used to store the hierarchical storage management program for multidimensional flight data; A processor is configured to implement the steps of the multidimensional flight data hierarchical storage management method as described in any one of claims 1-7 when executing the multidimensional flight data hierarchical storage management program.

10. A computer-readable storage medium storing a computer program, characterized in that, The readable storage medium stores a multi-dimensional flight data hierarchical storage management program, which, when executed by a processor, implements the steps of the multi-dimensional flight data hierarchical storage management method as described in any one of claims 1-7.

Images