Construction method and management system of multi-modal data set

Abstract

The invention provides a multi-modal data set construction method and management system, and relates to the technical field of data management and maintenance, and the method comprises the following steps: collecting the related feature information of data in a data set; systematized preprocessing is carried out on the related feature information; generating a storage quality index according to the related feature information score, and constructing a storage value evaluation system; when the storage space is lower than a space threshold value, collecting correction dynamic information of the data, and generating a storage correction coefficient through the correction dynamic information; and generating a storage optimization index based on the storage quality index and the storage correction coefficient, and re-evaluating the data. According to the method, a storage quality index and storage optimization index quantitative index system is introduced, a fine management closed loop which takes the data value as the center and actively adapts to business and resource changes is formed, and the utilization rate of storage resources and the cost effectiveness are remarkably improved while the data quality and the application efficiency are guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data management and maintenance, in particular to a multi-modal data set construction method and management system. BACKGROUND

[0002] The construction and management of multi-modal data sets is a systematic project aimed at converting heterogeneous data into high-quality AI assets. An efficient management system is the core support. When managing storage space, through systematic quantitative evaluation of data intrinsic quality, access frequency, business value, and storage cost, differentiated resource allocation can be performed on the data to improve storage resource utilization efficiency. When storage space is tight, through a scientific storage priority index evaluation model, optimized decisions can dynamically respond to changes in resource status, achieving adaptive balance between storage cost, data value, and compliance risk.

[0003] In the prior art, the publication number CN202311099876.1 discloses a multi-modal data set construction method and system. The technology includes: combining samples in multiple graphic-text data sets into a true data set, and generating a pseudo data set according to the samples in the true data set; mixing the samples in the true data set and the pseudo data set according to a specified mixing ratio, inputting the mixed samples into a multi-modal large model for graphic-text application training; updating the mixing ratio according to the loss value obtained by the graphic-text application training, repeating the mixing of samples in the true data set and the pseudo data set according to the updated mixing ratio, and repeating the graphic-text application training until the multi-modal large model reaches a specified precision; and using the mixed data obtained after training as a multi-modal data set.

[0004] However, in the above-mentioned prior art, the traditional method relies on manual experience and subjective selection during data collection, lacks a standardized quality evaluation system, and results in mixed formats, redundancy, and a large amount of invalid content in the data set, with uneven data quality. The storage management scheme usually adopts a homogeneous treatment strategy without differentiation, which cannot dynamically allocate resources according to the intrinsic value, access mode, and compliance requirements of the data, resulting in insufficient access performance of high-value data, while low-value data occupies expensive storage resources for a long time, and the overall cost-effectiveness is low.

[0005] The above information disclosed in the background section is only used to enhance the understanding of the background of the present disclosure, and therefore it can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0006] The application aims to provide a multi-modal data set construction method and management system to solve the problems in the background art. By introducing a storage quality index and a storage optimization index quantification index system, a fine management closed loop is formed, which is centered on data value and actively adapts to business and resource changes. While ensuring data quality and application efficiency, the storage resource utilization rate and cost-effectiveness are significantly improved.

[0007] To achieve the above-mentioned purpose, the application provides the following technical solutions:

[0008] S1: Collecting relevant feature information of data in the data set, the relevant feature information including format attribute, statistics of format of different files; content quality, statistics of invalid content and repeated content; access dynamics, statistics of access frequency and last access time; data features, statistics of keyness and retention date of data;

[0009] S2: Systematically preprocessing the relevant feature information, converting the relevant feature information into standardized and calculable high-quality features, providing input for constructing the storage quality index and driving automatic strategies, all preprocessing steps being performed through a unified ETL pipeline, and finally outputting a complete data asset feature table;

[0010] S3: Generating a storage quality index according to the relevant feature information score, constructing a storage value evaluation system through a systematic quantification method, evaluating the health status of the current storage data, establishing threshold standards and multi-level alarm mechanisms, and converting complex storage states into decision signals, so that when the storage quality index breaks through the threshold, the data is processed;

[0011] S4: When the storage space is lower than the space threshold, collecting the correction dynamic information of the data, the correction dynamic information including unit storage cost, data size, data value reference value and risk coefficient, and generating a storage correction coefficient through the correction dynamic information;

[0012] S5: Generating a storage optimization index based on the storage quality index and the storage correction coefficient, the storage optimization index re-evaluating the data storage state and serving as a core index for evaluating the storage state, and being directly used for determining whether the data needs to be processed, migrated or optimized, and for data management decision-making.

[0013] Further, the format attribute statistics the distribution and proportion of different file types to evaluate technical compatibility and processing needs; the content quality quantifies the effectiveness and redundancy of data by identifying empty files, error data and content redundancy; the access dynamics records and analyzes the access frequency, recent access time and change trend of data to measure the heat and activity of the data; and the data features focus on the business dimension, marking the key level and preset retention period of the data.

[0014] Further, based on the collected feature information, the system will start the pretreatment and quantitative scoring process, first, the original data is processed for abnormal value, to ensure the accuracy of the evaluation basis; then, according to the pre-defined scoring rules and models, the format attribute, content quality, access dynamics, data characteristics of each data object are respectively evaluated, finally, the system will output a structured data asset feature table, which records the original features and scores of each data object.

[0015] Further, the storage quality index is calculated by the following formula:

[0016]

[0017] Wherein:

[0018] is the storage quality index;

[0019] is the format attribute score;

[0020] is the content quality score;

[0021] is the access dynamics score;

[0022] is the data characteristics score;

[0023] are the weight coefficients of the format attribute score, content quality score, access dynamics score and data characteristics score respectively, which are used to reflect the influence degree of each feature information on the storage quality index, , and .

[0024] Further, when the storage quality index is lower than the threshold standard, it indicates that the overall storage health condition has entered the improvement or low efficiency interval, and the system must trigger the optimization demand; even if the storage quality index is not lower than the threshold standard, but the evaluation index of any one or more related feature information is lower than the independent threshold value of the feature information, it indicates that the data storage exists waste, and the system determines that the data storage exists local optimization demand, and automatically generates the targeted optimization task; at the same time, the decline trend of the storage quality index is monitored, and if the storage quality index continuously decreases in two consecutive periods, even if the storage quality index is still in the good interval, it should be determined as potential risk in advance, trigger the early warning and carry out the root cause analysis.

[0025] Furthermore, when the system detects that the available storage space is lower than a preset space threshold, it will automatically trigger a data reassessment process. At this time, the system will dynamically collect four key pieces of dynamic correction information to provide a basis for priority cleanup and migration decisions. The dynamic correction information includes:

[0026] Unit storage cost: This refers to the monthly cost per unit capacity of the storage medium where the data resides, and is used to measure the economics of storage.

[0027] Data size: The space occupied by the calculated data;

[0028] Data value reference value: The value of a unit of data estimated by combining business roles, the revenue generated, or the replacement cost;

[0029] Risk coefficient: Quantify the security, compliance and loss risks involved. The collected dynamic information will be incorporated into the cost-benefit and risk model to dynamically adjust the existing static storage quality index.

[0030] Furthermore, the storage correction coefficient is calculated using the following formula:

[0031]

[0032] in:

[0033] To store correction coefficients;

[0034] C represents the unit storage cost;

[0035] S represents the data size;

[0036] P represents the reference value of the data.

[0037] PF stands for Data Feature Scoring;

[0038] R is the risk coefficient.

[0039] Furthermore, the storage optimization index is calculated using the following formula:

[0040]

[0041] in It is a storage optimization index, ranging from [0,1].

[0042] Furthermore, the storage optimization index, by integrating the intrinsic quality of data and external environmental constraints, periodically and dynamically re-evaluates all data in the system. As a core indicator for quantifying storage status, the storage optimization index constructs a clear decision-making benchmark. By mapping the storage optimization index value to a preset threshold range, it directly and automatically determines the storage lifecycle operations required for each piece of data.

[0043] When the data storage optimization index is high, the system determines that the data is retained in the high-performance storage layer and ensures high availability. When the data storage optimization index drops to the medium range, it triggers an optimization instruction to migrate to a lower-cost storage medium. A key threshold is set. When the data storage optimization index falls below the key threshold, the system marks the data as to be cleaned up and deletes it.

[0044] A management system for a multimodal dataset, the system being used to execute the above-described method for constructing a multimodal dataset, the management system comprising:

[0045] Data feature acquisition module: used to automatically scan and collect four types of related feature information of data in the dataset: format attributes, content quality, access dynamics and data features, and output the collected raw information to the preprocessing module;

[0046] Data preprocessing module: It has a built-in unified ETL pipeline, which receives raw feature information from the acquisition module, cleans it, handles outliers, normalizes it, and integrates it, and finally outputs a complete data asset feature table.

[0047] Health assessment and monitoring module: Based on the scores in the data asset characteristic table, calculate the storage quality index of each data object, and set multi-level thresholds and trend monitoring to achieve global, local and forward-looking early warning;

[0048] Dynamic correction module: When storage is tight, it dynamically collects data cost, value, size and risk information, generates storage correction coefficients, and makes adaptive adjustments to environmental changes;

[0049] Decision execution module: Generates storage optimization index by combining storage quality index and storage correction coefficient, and automatically triggers data retention, migration and cleanup operations based on storage optimization index to achieve closed-loop management of the entire lifecycle.

[0050] Compared with the prior art, the beneficial effects of the present invention are:

[0051] By systematically collecting multi-dimensional information such as data format attributes, content quality, access dynamics, and data characteristics, quantitative analysis and comprehensive evaluation are conducted. Storage quality index and storage correction coefficient are introduced in one step, and finally a dynamic storage optimization index is generated as the core decision indicator. This can accurately assess the storage status of each piece of data, thereby automatically driving the entire lifecycle operation of data retention, migration, archiving, and cleanup, realizing intelligent optimization of storage resources and maximizing the value of data assets. Attached Figure Description

[0052] Fig. 1 This is a schematic diagram of the overall process flow of a method for constructing a multimodal dataset according to the present invention;

[0053] Fig. 2 This is a management system diagram for a multimodal dataset. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0055] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0056] Example:

[0057] Please see Figs. 1-2 The present invention provides a technical solution:

[0058] A method for constructing a multimodal dataset, comprising the following steps:

[0059] S1: Collect relevant feature information of the data within the dataset. The relevant feature information includes format attributes. The system automatically scans and analyzes the types of all files in the dataset, and calculates the distribution ratio and storage ratio of each type of file to evaluate the technical compatibility, processing complexity, and adaptation requirements of different analysis tools of the dataset. Content quality: The system detects and counts invalid or low-quality content in the data through predefined rules and algorithms, and identifies and counts completely duplicated and nearly duplicated content in the data dataset to quantify the validity and redundancy of the data.

[0060] Access dynamics: The system continuously tracks and records historical access patterns of data, including access frequency and last access timestamp. By analyzing this dynamic information, the system can quantify the popularity, activity, and value of data in current business operations. Data characteristics: Based on pre-set business rules, compliance requirements, and management strategies, the system labels key characteristics of data, including defining the data's criticality level, setting legally mandated and business-required retention and expiration dates, and associating relevant security and privacy tags, providing direct strategic basis for data lifecycle management.

[0061] S2: After the feature information is collected, the system will start a systematic preprocessing process, which aims to transform these heterogeneous information from diverse sources and in different formats into unified, standardized, and high-quality features that can be directly used for quantitative calculations. This process follows standardized data engineering principles, automatically executed through a structured ETL (Extract, Transform, Load) pipeline to ensure consistency and repeatability. Preprocessing includes several key steps: first, data cleaning, such as handling missing values, correcting obvious errors, and standardizing units of measurement; then, normalization operations, encoding text labels into numerical values, normalizing or binning continuous variables, and aligning and interpolating time-series data; finally, feature association and integration, merging feature information from different dimensions into complete data asset records based on unique identifiers; all intermediate states are logged, ensuring process traceability. After this series of meticulous processing steps, the system ultimately outputs a clearly structured, complete, and highly usable data asset feature table. This table records the cleaned, standardized feature values ​​and retains necessary metadata, providing clean, reliable, and directly accessible structured data input for subsequent construction of storage quality indices and driving automated storage optimization strategies.

[0062] S3: After completing the standardized preprocessing of feature information, the system enters the core evaluation stage. Based on the data asset feature table, a storage quality index is generated. The storage quality index is not a simple reflection of a single indicator, but rather a comprehensive evaluation system that uses a systematic and configurable quantitative model to weight and integrate four dimensions: data format attributes, content quality, access dynamics, and data characteristics. This system can comprehensively and objectively measure the value of data storage.

[0063] The system scores each data feature based on predefined scoring rules and algorithm models, and then synthesizes the final storage quality index through weighted geometric average. This index becomes the core benchmark for quantifying the health status of storage. The higher the storage quality index, the better the compliance, effectiveness, activity and policy alignment of data storage, and the better the overall health. The lower the storage quality index, the more likely there are risks of redundancy, idleness, non-compliance and inefficient management.

[0064] To achieve automated and intelligent management, the system establishes clear threshold standards and a multi-level alarm mechanism for the storage quality index. The thresholds are typically set in multiple key ranges, categorized as excellent, good, needing improvement, and inefficient. When the index drops and exceeds a certain threshold, the system can immediately transform the complex and multi-dimensional storage status degradation signals into clear and categorized actionable decision instructions.

[0065] Storage Quality Index (SQI) is calculated using the following formula:

[0066]

[0067] in:

[0068] Storage quality index;

[0069] The format attribute is scored by evaluating the format of the data. Increasing the numerical value will positively increase the SQI value.

[0070] The content quality is scored by assessing the percentage of invalid and duplicate content. Increasing the numerical value will positively increase the SQI value.

[0071] The access dynamic score is obtained by combining the frequency of data access and the time interval between the last access. Increasing the numerical value will positively increase the SQI value.

[0072] The data features are scored by combining the criticality level score and the time period score. Increasing the numerical value will positively increase the SQI value.

[0073] These are weighting coefficients for the format attribute score, content quality score, access dynamics score, and data feature score, respectively. These weighting coefficients reflect the degree of influence of each feature information on the storage quality index. ,and The importance of data features and their lifecycle are key factors in determining whether to optimize the data, so their weighting coefficients should be greater than other weighting coefficients.

[0074] When the storage quality index falls below the preset threshold, it indicates that the overall health of data storage has entered a range of needing improvement or inefficiency. At this point, the system must immediately trigger a global optimization requirement. The optimization method is for the system to automatically match and launch preset special governance projects to systematically solve the fundamental problems. At the same time, a detailed diagnostic report is generated and submitted to the administrator, and manual intervention is initiated for review to decide on a large-scale governance plan.

[0075] Even if the storage quality index is not lower than the threshold standard, if the evaluation index of any one or more related feature information is lower than the independent threshold of the feature information, it indicates that there is resource waste or management defects in a specific dimension. Based on this, the system accurately determines that there is a local optimization need and automatically generates targeted optimization tasks.

[0076] The system continuously monitors the change curve of the storage quality index. If it finds that the storage quality index shows a continuous downward trend in two consecutive evaluation periods, even if the absolute value is still in the good range, the system will identify it as a potential risk signal in advance. Instead of waiting for the threshold to be broken, the system will actively trigger a trend warning and simultaneously start root cause analysis to trace the main characteristic information that caused the decline. Preventive optimization will be carried out by adjusting strategies and optimizing configurations.

[0077] S4: When the system detects through the real-time resource monitoring module that the current available storage space capacity is continuously lower than the preset space threshold, the system will immediately and automatically trigger and execute a dynamic data reassessment process. Under resource constraints, the data value will be recalibrated, providing accurate and differentiated decision-making basis for data cleaning, migration and compression optimization operations.

[0078] The system will collect and integrate four key dynamic information points in real time to construct a dynamic decision-making framework oriented towards cost, value, and risk. The dynamic information points include: unit storage cost, where the system automatically identifies the storage media level where the data object is currently located and calls the cost management module to obtain the latest monthly storage cost per GB of that media, thereby accurately measuring the direct economic burden of retaining data, which is the core economic input for optimization decisions; data size, where the system accurately calculates the actual physical storage space currently occupied by the data object, thereby assessing the direct quantitative factor of the data's contribution to the overall space shortage situation; and data size, where the system accurately calculates the actual physical storage space occupied by the data object, thereby assessing the direct quantitative factor of the data's contribution to the overall space shortage situation.

[0079] The data value reference value system dynamically estimates the unit value of data by comprehensively analyzing multiple information sources. This includes assessing the core role of data in business processes, analyzing the indirect business benefits associated with its historical access patterns, and estimating the replacement costs required to reacquire and rebuild data after it is lost. The data value reference value aims to transform the business importance of data into a comparable economic equivalent.

[0080] The risk coefficient is a quantitative score used by the system to assess the security, compliance, and business continuity risks associated with the data. These risks include: the risk of security breaches corresponding to the data sensitivity level, the risk of violating compliance requirements of industry regulations, and the risk of business interruption due to data unavailability. The risk coefficient serves as a discount factor, reflecting the potential negative impacts that data disposal may bring.

[0081] The storage correction factor is calculated using the following formula:

[0082]

[0083] in:

[0084] To store correction coefficients;

[0085] C represents the unit storage cost, and S represents the data size. The unit storage cost and data size constitute cost pressure, which is negatively correlated with the storage correction coefficient. The higher the value, the lower the storage priority.

[0086] P represents the data value reference value, which constitutes the value support and is positively correlated with the storage correction coefficient, reflecting the upper limit of data investment costs.

[0087] PF stands for Data Feature Score, which serves as a compliance baseline and is proportional to the storage correction coefficient.

[0088] R is the risk coefficient. The higher the risk coefficient, the greater the risk of the data, and the lower the value of the storage correction coefficient will be.

[0089] S5: Based on the existing storage quality index and storage correction coefficient, the system recalculates and generates a brand-new, decision-oriented comprehensive index, the storage optimization index, through a preset synthesis algorithm. The storage optimization index is not a simple re-scoring of the data, but rather represents a dynamic re-evaluation of the value and priority of data storage under specific resource and environmental constraints.

[0090] The storage optimization index is calculated using the following formula:

[0091]

[0092] in: It is a storage optimization index, ranging from [0,1].

[0093] The calculation of the storage optimization index essentially couples the intrinsic quality of data, as reflected by SQI, with the external constraints, including cost, space, value, and risk, as reflected by SDF. When storage space is ample and cost pressure is low, SDF approaches 1, and MDSOI mainly reflects the intrinsic quality of data. When storage space is tight, SDF will be dynamically adjusted downward according to the cost-benefit and risk model, thereby significantly lowering the MDSOI value of data with high cost, low value, and low risk, making it stand out as an object to be optimized in the overall evaluation.

[0094] The generated Storage Optimization Index (MDSOI) is established as the core, unified decision-making indicator for the system to assess storage status during resource-sensitive periods. The MDSOI provides a benchmark that can be directly used to compare the priorities of all data. By mapping the MDSOI value to predefined, clear key threshold ranges, the system can automatically and unambiguously determine the storage lifecycle operations required for each piece of data.

[0095] High MDSOI range: This is considered high-value core data and should be retained in a high-performance storage layer with high availability guaranteed; Medium MDSOI range: This is considered data that can be optimized and should be automatically compressed or migrated to a lower-cost storage medium; Low MDSOI range: This is considered low-priority or negative-benefit data and should be clearly marked as pending cleanup, and can be further automatically deleted or deeply archived.

[0096] The generation and application of the storage optimization index realizes a key shift in data management decision-making from static evaluation to dynamic optimization, and from quality-oriented to cost-value-risk-integrated orientation. It transforms complex business rules, resource status, and economic models into a single, actionable numerical signal, providing administrators with direct, quantitative, and critical core decision-making basis for data retention, migration, or cleanup in complex scenarios, driving storage management to achieve true intelligence and lean management.

[0097] This embodiment also provides a multimodal dataset management system for executing the above-described multimodal dataset construction method. The multimodal dataset management system includes:

[0098] Data feature acquisition module: used to automatically scan and collect four types of related feature information of data in the dataset: format attributes, content quality, access dynamics and data features, and output the collected raw information to the preprocessing module;

[0099] Data preprocessing module: It has a built-in unified ETL pipeline, which receives raw feature information from the acquisition module, cleans it, handles outliers, normalizes it, and integrates it, and finally outputs a complete data asset feature table.

[0100] Health assessment and monitoring module: Based on the scores in the data asset characteristic table, calculate the storage quality index of each data object, and set multi-level thresholds and trend monitoring to achieve global, local and forward-looking early warning;

[0101] Dynamic correction module: When storage is tight, it dynamically collects data cost, value, size and risk information, generates storage correction coefficients, and makes adaptive adjustments to environmental changes;

[0102] Decision execution module: Generates storage optimization index by combining storage quality index and storage correction coefficient, and automatically triggers data retention, migration and cleanup operations based on storage optimization index to achieve closed-loop management of the entire lifecycle.

[0103] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0104] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0105] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0106] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for constructing a multimodal dataset, characterized in that, The specific steps include: S1: Collect relevant feature information of the data within the dataset. The relevant feature information includes format attributes, which statistically analyze the formats of different files; content quality, which statistically analyzes invalid and duplicate content; access dynamics, which statistically analyzes access frequency and last access time; and data characteristics, which statistically analyze the keyness of the data and the retention date. S2: Systematically preprocess the relevant feature information, transforming it into standardized, computable, high-quality features to provide input for building storage quality indices and driving automation strategies. All preprocessing steps are carried out through a unified ETL pipeline, ultimately outputting a complete and correlated data asset feature table. S3: Generate a storage quality index based on relevant feature information scores, construct a storage value assessment system through a systematic quantitative method, assess the current health status of stored data, establish threshold standards and multi-level alarm mechanisms, transform complex storage status into decision signals, and process the data when the storage quality index exceeds the threshold; S4: When the storage space is lower than the space threshold, collect dynamic information for data correction. The dynamic information for correction includes unit storage cost, data size, data value reference value and risk coefficient. A storage correction coefficient is generated through the dynamic information for correction. S5: Generate a storage optimization index based on the storage quality index and the storage correction coefficient. The storage optimization index re-evaluates the data storage status and serves as the core indicator for evaluating the storage status. It is directly used to determine whether the data needs to be processed, migrated, or optimized, and is used in the data management decision-making process.

2. The method and management system for constructing a multimodal dataset according to claim 1, characterized in that: The format attributes statistically analyze the distribution and proportion of different file types to assess technical compatibility and processing requirements; content quality quantifies the validity and redundancy of data by identifying empty files, erroneous data, and content duplication; access dynamics records and analyzes the access frequency, recent access time, and change trends of data to measure data popularity and activity; data features focus on business dimensions, marking the key level of data and preset retention period.

3. The method for constructing a multimodal dataset according to claim 1, characterized in that: Based on the collected feature information, the system will initiate a preprocessing and quantitative scoring process. First, outlier handling will be performed on the raw data to ensure the accuracy of the assessment basis. Subsequently, based on predefined scoring rules and models, the system evaluates the scores of four features for each data object: format attributes, content quality, access dynamics, and data characteristics. Finally, the system outputs a structured data asset feature table, which fully records the original features and scores of each data object.

4. The method for constructing a multimodal dataset according to claim 1, characterized in that: The storage quality index is calculated using the following formula: ； in: Storage quality index; Rate the format attributes; Rate the content quality; Dynamic rating for access; Score the data features; These are weighting coefficients for the format attribute score, content quality score, access dynamics score, and data feature score, respectively. These weighting coefficients reflect the degree of influence of each feature information on the storage quality index. ,and .

5. The method for constructing a multimodal dataset according to claim 4, characterized in that: When the storage quality index falls below the threshold, it indicates that the overall storage health status has entered a range requiring improvement or is inefficient, and the system must trigger optimization. Even if the storage quality index is not below the threshold, if the evaluation index of any one or more related feature information is below the independent threshold of the feature information, it indicates that data storage is wasted. The system determines that there is a local optimization need for data storage and automatically generates targeted optimization tasks. At the same time, the system monitors the downward trend of the storage quality index. If the storage quality index continues to decline for two consecutive periods, even if the storage quality index is currently still in the good range, it should be identified as a potential risk in advance, triggering an early warning and conducting root cause analysis.

6. The method for constructing a multimodal dataset according to claim 1, characterized in that: When the system detects that the available storage space is lower than the preset space threshold, it will automatically trigger the data reassessment process. At this point, the system will dynamically collect four key pieces of corrective dynamic information to provide a basis for priority cleanup and migration decisions. This corrective dynamic information includes: Unit storage cost: This refers to the monthly cost per unit capacity of the storage medium where the data resides, and is used to measure the economics of storage. Data size: The space occupied by the calculated data; Data value reference value: The value of a unit of data estimated by combining business roles, the revenue generated, or the replacement cost; Risk coefficient: Quantify the security, compliance and loss risks involved. The collected dynamic information will be incorporated into the cost-benefit and risk model to dynamically adjust the existing static storage quality index.

7. The method for constructing a multimodal dataset according to claim 1, characterized in that: The storage correction factor is calculated using the following formula: ； in: To store correction coefficients; Cost per unit of storage; For data size; As a reference value for data; Score the data features; This represents the risk coefficient.

8. The method for constructing a multimodal dataset according to claim 1, characterized in that: The storage optimization index is calculated using the following formula: ； in It is a storage optimization index, ranging from [0,1].

9. The method for constructing a multimodal dataset according to claim 1, characterized in that: The storage optimization index, by integrating the intrinsic quality of data and external environmental constraints, periodically and dynamically re-evaluates all data in the system. As a core indicator for quantifying storage status, the storage optimization index constructs a clear decision-making benchmark. By mapping the storage optimization index value to a preset threshold range, it directly and automatically determines the storage lifecycle operations required for each piece of data. When the data storage optimization index is high, the system determines that the data is retained in the high-performance storage layer and ensures high availability. When the data storage optimization index drops to the medium range, an optimization instruction is triggered to migrate to a lower-cost storage medium. When a critical threshold is set, the system will mark the data as needing to be cleaned up and delete it when the data storage optimization index falls below the critical threshold.

10. A management system for multimodal datasets, characterized in that: The management system for the multimodal dataset is used to execute the method for constructing a multimodal dataset as described in any one of claims 1-9, wherein the management system for the multimodal dataset includes: Data feature acquisition module: used to automatically scan and collect four types of related feature information of data in the dataset: format attributes, content quality, access dynamics and data features, and output the collected raw information to the preprocessing module; Data preprocessing module: It has a built-in unified ETL pipeline, which receives raw feature information from the acquisition module, cleans it, handles outliers, normalizes it, and integrates it, and finally outputs a complete data asset feature table. Health assessment and monitoring module: Based on the scores in the data asset characteristic table, calculate the storage quality index of each data object, and set multi-level thresholds and trend monitoring to achieve global, local and forward-looking early warning; Dynamic correction module: When storage is tight, it dynamically collects data cost, value, size and risk information, generates storage correction coefficients, and makes adaptive adjustments to environmental changes; Decision execution module: Generates storage optimization index by combining storage quality index and storage correction coefficient, and automatically triggers data retention, migration and cleanup operations based on storage optimization index to achieve closed-loop management of the entire lifecycle.

Images