Inventory method and system based on data assets and storage medium

Abstract

The invention relates to the technical field of data management, in particular to a checking method and system based on data assets and a storage medium. Randomly selecting and obtaining a first subject from the plurality of subjects, and collecting a first data set of the first subject as an accumulated data set; continuing to collect a second data set of the second subject, and analyzing a second data value factor set of the second data set according to the accumulated data set; according to the proportion of the data of each main body in the accumulative data set, configuring at least one classification data extraction coefficient, extracting at least one classification data set, constructing an exception classifier, performing exception classification on a plurality of pieces of second data, and obtaining a second data credible factor set; and screening to obtain data greater than or equal to a data screening factor threshold, adding the data into an accumulated data set, and continuously screening to obtain an accumulated data set of all subjects as a data asset inventory result. The technical effect of improving the efficiency and quality of data asset checking is achieved.

Description

Technical Field

[0001] The present invention relates to the field of data management technology, and in particular to a data asset-based inventory method, system, and storage medium. Background Art

[0002] In today's data-driven era, data assets have become a critical strategic resource for organizations. Effectively inventorying data assets is fundamental to maximizing data value. With the rapid development of information technology, the amount of data in organizations has exploded. Data comes from a wide range of sources, involves multiple entities, and has complex and diverse data types. Against this backdrop, accurately identifying valuable information from massive amounts of data from multiple entities, eliminating useless and anomalous data, and achieving a comprehensive and accurate inventory of data assets has become a critical issue for organizations. Traditional data inventory methods often lack objectivity, conduct unscientific assessments of data value, and may be biased in initial data selection, leading to distorted subsequent analysis results. They also rely heavily on a single dimension or subjective judgment, making it difficult to accurately quantify the actual value of the data. Consequently, they face technical challenges in terms of efficiency and quality in data asset inventory. Summary of the Invention

[0003] The present invention aims to solve the technical problem of low efficiency and quality of data asset inventory in the prior art by providing a data asset-based inventory method, system and storage medium.

[0004] The technical solution of the present invention to solve the above technical problems is as follows: In a first aspect, the present invention provides an inventory method based on data assets, comprising: randomly selecting a first subject from multiple subjects, collecting a first data set of the first subject as a cumulative data set, wherein the cumulative data set is updated as the subject data is traversed; continuing to collect a second data set of the second subject, and analyzing a second data value factor set of the second data set based on the cumulative data set; configuring at least one classification data extraction coefficient based on the proportion of data of each subject in the cumulative data set, extracting at least one classification data set, constructing an anomaly classifier, classifying multiple second data as anomalies, and obtaining a second data credibility factor set; calculating a second data screening factor set based on the second data value factor set and the second data credibility factor set, screening to obtain data greater than or equal to a data screening factor threshold, adding the data to the cumulative data set, and continuing to screen to obtain the cumulative data sets of all subjects as the data asset inventory result.

[0005] Optionally, a first subject is randomly selected from multiple subjects, and a first data set of the first subject is collected as a cumulative data set, including: obtaining multiple subjects to be subject to data asset inventory; selecting a first subject from the multiple subjects, and collecting the first data set of the first subject as a cumulative data set, wherein the cumulative data set is updated as the subject data is traversed and collected.

[0006] Optionally, continue to collect the second data set of the second subject, and analyze the second data value factor set of the second data set based on the accumulated data set, including: collecting the second data set of the second subject, wherein the second data set includes multiple second data; based on the accumulated data set, analyze the second data value factors of the multiple second data in the second data set to obtain the second data value factor set.

[0007] Among them, according to the accumulated data set, the second data value factors of multiple second data in the second data set are analyzed to obtain a second data value factor set, including: calculating the similarity between each second data in the second data set and all data sets in the accumulated data set, and calculating the average to obtain a second data similarity set; according to the second data similarity set, the second data value factor set is calculated.

[0008] Optionally, according to the proportion of data of each subject in the cumulative data set, at least one classification data extraction coefficient is configured, at least one classification data set is extracted, an anomaly classifier is constructed, and multiple second data are classified as anomalies to obtain a second data credibility factor set, including: calculating the proportion of data of each subject in the cumulative data set as at least one classification data extraction coefficient; obtaining a preset data extraction quantity, and calculating and obtaining at least one classification data extraction amount in combination with the at least one classification data extraction coefficient; randomly extracting data of at least one subject in the cumulative data set according to the at least one classification data extraction quantity to obtain at least one classification data set; using the at least one classification data set, an anomaly classifier is constructed, and the second data is classified as anomalies to obtain a second data credibility factor set.

[0009] Among them, using the at least one classification data set to construct an anomaly classifier, performing anomaly classification on multiple second data, and obtaining a second data credibility factor set, including: combining the at least one classification data set to obtain an anomaly classification data set, randomly selecting first anomaly classification data, and constructing a layer of classification nodes, wherein the layer of classification nodes performs binary classification on the input data and marks the data as abnormal data when it is classified as a single data; continuing to construct to obtain multiple layers of classification nodes to obtain an anomaly classifier; inputting multiple second data into the anomaly classifier, obtaining the number of node layers in which each second data is classified as abnormal data, calculating the ratio to the maximum number of node layers, obtaining multiple second data credibility factors, and integrating to obtain the second data credibility factor set.

[0010] Optionally, based on the second data value factor set and the second data credibility factor set, a second data screening factor set is calculated, data greater than or equal to the data screening factor threshold is screened and added to the cumulative data set, and the cumulative data set of all subjects is continued to be screened as the data asset inventory result, including: based on the second data value factor set and the second data credibility factor set, a second data screening factor set is calculated; second data with a second data screening factor greater than or equal to the data screening factor threshold is screened and added to the cumulative data set; and data of other subjects are continued to be collected and screened to obtain the cumulative data set of all subjects as the data asset inventory result.

[0011] In a second aspect, the present invention provides an inventory system based on data assets, comprising: an accumulated data collection module, configured to randomly select a first subject from a plurality of subjects, collect a first data set of the first subject as an accumulated data set, wherein the accumulated data set is updated as the data of the subjects are traversed; a value factor analysis module, configured to continue collecting a second data set of a second subject, and analyze a second data value factor set of the second data set based on the accumulated data set; a data credibility analysis module, configured to configure at least one classification data extraction coefficient based on the proportion of data of each subject in the accumulated data set, extract at least one classification data set, construct an anomaly classifier, perform anomaly classification on the plurality of second data, and obtain a second data credibility factor set; The cumulative data screening module is used to calculate the second data screening factor set based on the second data value factor set and the second data credibility factor set, screen and obtain data greater than or equal to the data screening factor threshold, add it to the cumulative data set, and continue to screen and obtain the cumulative data set of all entities as the data asset inventory result.

[0012] In a third aspect, the present application provides a storage medium storing a second computer software program, which, when executed by a processor, implements an inventory method based on data assets in the first aspect.

[0013] By implementing the present invention, it is possible to randomly select a first subject from multiple subjects and collect a first data set of the first subject as a cumulative data set. The cumulative data set is updated as the data of the collected subjects are traversed. Randomly selecting the first subject can avoid bias caused by human selection, making the initial cumulative data set more objective and laying a fair foundation for subsequent data processing. The cumulative data set is dynamically updated during the collection process and can integrate new data in real time, ensuring that the data set can promptly reflect the data status of the collected subjects and provide the latest reference basis for subsequent analysis. By implementing the present invention, it is possible to continue collecting the second data set of the second subject and analyze the second data value factor set of the second data set based on the accumulated data set. Because the accumulated data set contains valuable data information collected previously, analyzing the value factor of the second data set with the accumulated data set as a reference can make the analysis results more targeted and relevant. By analyzing the second data value factor set obtained, the value of each piece of second data can be quantified, providing an important value judgment basis for subsequent data screening. By implementing the present invention, it is possible to configure at least one classification data extraction coefficient based on the proportion of data of each subject in the cumulative data set, extract at least one classification data set, construct an anomaly classifier, perform anomaly classification on multiple second data, and obtain a second data credibility factor set. Configuring the extraction coefficient based on the proportion of subject data can make the extracted classification data set more representative, reflect the weight of different subject data in the cumulative data set, and improve the construction quality of the anomaly classifier. Constructing the anomaly classifier to perform anomaly classification on the second data can effectively identify abnormal data, and the obtained credibility factor set can quantify the credibility of each second data, providing a reliability judgment standard for data screening; By implementing the present invention, it is possible to calculate and obtain the second data screening factor set based on the second data value factor set and the second data credibility factor set, screen and obtain data that is greater than or equal to the data screening factor threshold, add the data to the cumulative data set, and continue to screen and obtain the cumulative data set of all subjects as the data asset inventory result. By screening based on the factors of the two dimensions of value and credibility, the data can be evaluated more comprehensively, ensuring that the data added to the cumulative data set has both high value and high credibility, improving the quality of the cumulative data set, screening all subjects and forming inventory results, and realizing a systematic inventory of data assets, ensuring the integrity and accuracy of the inventory results, and providing a reliable basis for the management and utilization of data assets.

[0014] In summary, by implementing the present invention, the technical effect of improving the efficiency and quality of data asset inventory can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flowchart of a data asset inventory method provided by the present invention; Figure 2 A schematic diagram of the structure of a data asset inventory system provided by the present invention; Figure 3 A schematic diagram of a storage medium provided by the present invention.

[0016] In the accompanying drawings, the components represented by the reference numerals are as follows: Cumulative data collection module 11 , value factor analysis module 12 , data credibility analysis module 13 , cumulative data screening module 14 , storage medium 400 , first computer program 410 . DETAILED DESCRIPTION

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0018] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include one or more features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0019] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or explanation". Any embodiment of the present invention described as "for example" is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed herein.

[0020] Example 1, as Figure 1 As shown, an embodiment of the present invention provides an inventory method based on data assets, including: In step S100 of the embodiment of the present application, a first subject is randomly selected from multiple subjects, and a first data set of the first subject is collected as an accumulated data set, including: Obtain multiple entities for data asset inventory; A first subject among the multiple subjects is selected, and a first data set of the first subject is collected as an accumulated data set, wherein the accumulated data set is updated as the subject data is traversed and collected.

[0021] In the present embodiment, the purpose of step S100 is to establish an objective, dynamic initial baseline for the entire data asset inventory process. On the one hand, by randomly selecting the first subject, initial data bias caused by human selection is avoided, ensuring that the starting point of the cumulative dataset is representative, and reducing errors in subsequent analysis caused by initial data bias. On the other hand, it is clear that the cumulative dataset is continuously updated as new subject data is collected, allowing the dataset to integrate new information in real time, providing a continuously improved reference basis for subsequent value assessment and anomaly detection of other subject data.

[0022] Assume that a factory has 100 pieces of production equipment of the same model, namely the multiple entities mentioned above. The operating data of each piece of equipment, such as temperature, speed, energy consumption, fault code, etc., are data assets. These data can be used to train equipment operation and maintenance models, and can even be sold as sample data to companies in the same industry. They have certain commercial value, so a data asset inventory is required.

[0023] First, you need to obtain multiple entities for data asset inventory. Assume that the multiple entities here are 100 production equipment in a factory, which are numbered 1-100.

[0024] Next, a first subject from the multiple subjects needs to be selected, and a first data set of the first subject is collected as a cumulative data set. The cumulative data set is updated as the subject data is collected. In the above example, the first subject is randomly selected from 100 devices using a random number generator or other method. Assume that device 37 is randomly selected.

[0025] Then, the first data set of device 37 is collected, including all its operating data in the past three months, such as 2,000 temperature records, 500 energy consumption data, 20 fault codes, etc. These data are packaged into "cumulative data set V1.0", that is, the initial cumulative data set.

[0026] Then continue to collect data sets of other entities such as device 89 and device 5, and repeat the above process. After the data of each new device is screened, the qualified part will be added to the cumulative data set, so that it is continuously updated, such as V3.0, V4.0..., and finally covers the effective operation data of all 100 devices.

[0027] In step S200 of the embodiment of the present application, the second data set of the second subject is continuously collected, and a second data value factor set of the second data set is analyzed based on the accumulated data set, including: collecting a second data set of a second subject, wherein the second data set includes a plurality of second data; According to the accumulated data set, second data value factors of a plurality of second data in the second data set are analyzed to obtain a second data value factor set.

[0028] In the embodiment of the present application, the core purpose of collecting the second data set of the second subject is to provide a specific analysis object for the subsequent analysis of the data value, and it is a key link in connecting the cumulative data set with the second data value factor set. Specifically, on the basis of the acquired first subject data, the data of the second subject is collected, and more subjects are gradually covered to provide a basis for a comprehensive inventory of data assets and avoid incomplete inventory due to missing subjects. The multiple second data contained in the second data set are the direct objects for the subsequent calculation of the "second data value factor set". Only by clarifying the specific multiple data can the value of each data be analyzed based on the cumulative data set, and a refined evaluation of the second subject data can be achieved.

[0029] First, the second subject needs to be determined. Among the multiple subjects whose data assets are to be inventoried, the second subject after the first subject is selected. For example, in the example of 100 devices, device 15 is selected as the second subject after the randomly selected device 37.

[0030] All relevant data generated by the second subject is then collected to form a second data set. Taking a device as an example, the second data set may include multiple pieces of operating data for the device 15, such as temperature data at different time points, such as "30°C" and "32°C"; energy consumption data, such as "100kWh" and "105kWh"; and fault codes, such as "E01" and "E02." Each piece of data is considered independent second data.

[0031] During the collection process, it is necessary to ensure that the second data set covers multiple categories and multiple data of the subject, rather than single data or one-sided data, to ensure that subsequent value analysis can cover the full picture of the second subject data.

[0032] For example, in the above example, the first agent is device 37, and the accumulated data set includes 2000 temperature records and 500 energy consumption data records for three months. The second agent selects device 15 and collects three months of its operating data, which is the second data set. This data set includes 1800 temperature records, 450 energy consumption data records, and 15 fault codes. These individual temperature, energy consumption, and fault code records are collectively referred to as "second data."

[0033] These multiple pieces of second data will serve as the basis for subsequent calculations of similarity and data value factors with the accumulated data set, for example, analyzing the similarity between a certain "E01" fault code of device 15 and a similar code in the data of device 37, and then evaluating its data value.

[0034] In step S200 of the embodiment of the present application, second data value factors of a plurality of second data items in the second data set are analyzed based on the accumulated data set to obtain a second data value factor set, including: Calculating the similarity between each second data in the second data set and all data sets in the cumulative data set, and calculating the average to obtain a second data similarity set; A second data value factor set is calculated based on the second data similarity set.

[0035] In this step, the core purpose of calculating the second data value factor set is to quantitatively evaluate the relative value of each piece of data in the second subject, and to provide a judgment standard based on the data value dimension for subsequent data screening. The logic is that the lower the similarity between the second data and the cumulative data set, the more new information or special features it contains, and the higher the potential value; conversely, high similarity may indicate repeated or redundant data with lower value. The mean similarity can objectively reflect the uniqueness of each piece of second data. Further converting the similarity set into a value factor set can convert the abstract concept of data value into a specific numerical value, making the value of different data comparable and screenable, and providing a clear basis for subsequent comprehensive credibility factor data evaluation.

[0036] First, the similarity between each piece of second data in the second dataset and all data in the cumulative dataset needs to be calculated. Specifically, for each piece of second data in the second dataset, its similarity with all data in the cumulative dataset needs to be calculated. For example, if the second data is "temperature 35°C + speed 1500 rpm" for a device, its similarity needs to be calculated with all temperature and speed combinations of device 37 in the cumulative dataset.

[0037] For example, if the similarity between a piece of data from device 15 ("E01 fault + temperature 40°C") and the 100 pieces of related fault data in the cumulative data set from device 37 is calculated, the average similarity is 30%. Therefore, the similarity of this piece of data is 30% and is included in the similarity set. The similarity between each piece of data in the second data set and all the data sets in the cumulative data set is calculated, and the average is calculated to obtain the second data similarity set.

[0038] Among them, the specific similarity calculation method, for example, can first decompose each data into two key feature dimensions, namely the fault code as a categorical variable and the temperature as a numerical variable. The temperature needs to be standardized according to the normal operating temperature range of the equipment. Assuming that the normal operating temperature range of the equipment is 30℃~50℃, the temperature standardization formula is, standardized temperature = (actual temperature-30) / (50-30), therefore, the standardized temperature of device 15 is (40-30) / (50-30)=0.5. Furthermore, it is necessary to calculate the temperature difference between the two data, that is, the absolute difference in the standardized temperature. Assuming that the cumulative data used for calculation here is E01 fault + temperature 38℃, then its standardized temperature is 0.4, then the temperature difference between the two data = |0.5-0.4|=0.1.

[0039] As for the fault code, it can be measured by the matching degree. If the fault codes of the two sets of data are the same, that is, both are E01, the matching degree is 1; otherwise, it is 0.

[0040] Furthermore, based on the degree of reliance on the fault code matching and the temperature difference during actual implementation, weights W1 and W2 are assigned to the fault code matching and the temperature difference, and the similarity is obtained by weighted summation. The sum of W1 and W2 is 1. That is, similarity = W1 × fault code matching + W2 × (1 - |temperature difference|). Assuming W1 is 0.7 and W2 is 0.3, similarity = 0.7 × 1 + 0.3 × (1 - 0.1) = 0.7 + 0.27 = 0.97, or 97%.

[0041] Then, based on the second data similarity set, the average similarity is converted into a value factor using a preset formula. Generally, similarity and value factor are negatively correlated: the lower the similarity, the higher the value factor. For example, value factor = 1 - average similarity. If the average similarity is 30%, the value factor is 70%. The value factors of all second data are then combined into a second data value factor set, completing the quantification of the value of each piece of second data.

[0042] In step S300 of the embodiment of the present application, at least one classification data extraction coefficient is configured based on the proportion of data of each subject in the accumulated data set, at least one classification data set is extracted, an anomaly classifier is constructed, and anomaly classification is performed on the plurality of second data to obtain a second data credibility factor set, including: Calculating the proportion of data of each subject in the accumulated data set as at least one classification data extraction coefficient; Obtaining a preset data extraction quantity, and calculating at least one classification data extraction amount based on the at least one classification data extraction coefficient; randomly extracting data of at least one subject from the accumulated data set according to the at least one classification data extraction quantity to obtain at least one classification data set; An anomaly classifier is constructed using the at least one classification data set, and an anomaly classification is performed on the second data to obtain a second data credibility factor set.

[0043] In the embodiment of the present application, the purpose of this step is to construct a representative abnormality classifier training sample to lay the foundation for the subsequent identification of the abnormality of the second data and calculation of the credibility factor.

[0044] The logic is that by configuring the extraction coefficient based on the proportion of subject data, the extracted classification dataset can reflect the weights of different subjects in the cumulative dataset. For example, if the proportion of a subject data is high, its sample size in the classification dataset will also be correspondingly larger, avoiding sample bias that may cause the abnormal classifier to deviate from the recognition of specific subject data. In addition, by calculating the amount of classification data extracted based on the preset number of data extractions, the size of the classification dataset can be controlled while ensuring sample representativeness. This avoids insufficient generalization ability of the classifier due to too few samples, and avoids the increase in computational costs due to too many samples, ensuring the efficiency and accuracy of the construction of the abnormal classifier.

[0045] First, we need to calculate the proportion of each subject's data in the cumulative dataset to serve as at least one classification data extraction coefficient. Specifically, we need to calculate the proportion of each subject's data volume to the total data volume in the cumulative dataset, i.e., the subject's data proportion, and use this proportion directly as the classification data extraction coefficient for the corresponding subject.

[0046] For example, the cumulative data set contains 1000 data items from device 37 and 500 data items from device A, with a total data volume of 1500 items. Then the extraction coefficient of device 37 = 1000 / 1500 ≈ 66.7%, and the extraction coefficient of device A = 500 / 1500 ≈ 33.3%.

[0047] Next, we need to obtain a preset number of data to be extracted and, combined with the at least one classification data extraction coefficient, calculate at least one classification data extraction quantity. We first need to determine the total number of data to be extracted. For example, to build an anomaly classifier, we plan to extract 300 data items from the cumulative dataset as samples. We then multiply the preset total number by the extraction coefficient for each subject to obtain the classification data extraction quantity for that subject, i.e., the number of samples that should be extracted for that subject.

[0048] For example, if the preset number of data to be extracted is 300 pieces of data, the data extracted by device 37 = 300 × 66.7% ≈ 200 pieces of data, and the data extracted by device A = 300 × 33.3% ≈ 100 pieces of data.

[0049] Finally, according to the at least one classification data extraction quantity, data of at least one subject is randomly extracted from the accumulated data set to obtain at least one classification data set.

[0050] For example, according to the calculated classification data extraction amount, a corresponding amount of data is randomly extracted from each subject data in the cumulative data set, for example, 200 data are randomly extracted from 1000 data of device 37, and 100 data are randomly extracted from 500 data of device A.

[0051] All extracted data are integrated to form a classification data set, that is, a set of 300 data in the above example, which is used to subsequently build an anomaly classifier.

[0052] In step S300 of the embodiment of the present application, the at least one classification data set is used to construct an anomaly classifier, and anomaly classification is performed on the plurality of second data to obtain a second data credibility factor set, including: Combining the at least one classification data set to obtain an abnormal classification data set, randomly selecting first abnormal classification data, and constructing a layer of classification nodes, wherein the layer of classification nodes performs binary classification on the input data and marks the data as abnormal data when the data is classified as a single data; Continue to build and obtain multi-layer classification nodes to obtain anomaly classifiers; Input multiple second data into the anomaly classifier, obtain the node layer number of each second data being classified as anomaly data, calculate the ratio to the maximum node layer number, obtain multiple second data credibility factors, and integrate them to obtain a second data credibility factor set.

[0053] In the embodiment of the present application, the purpose of performing abnormal classification on multiple second data and obtaining a set of second data credibility factors is to quantitatively evaluate the credibility of each data in the second subject and provide a judgment standard based on the "reliability" dimension for subsequent data screening.

[0054] The logic is to build an anomaly classifier through multiple layers of classification nodes, gradually refining the judgment of data anomalies, improving the accuracy of abnormal data identification, and avoiding misjudgments caused by the one-sidedness of single-layer classification. The credibility of each piece of data is quantified by the ratio of the number of node layers when the data is judged to be abnormal to the maximum number of layers. For example, data that is judged to be abnormal only at deeper nodes has a higher credibility factor. This makes the credibility index quantifiable and comparable, providing a reliable basis for subsequent comprehensive value factor screening of data.

[0055] First, it is necessary to combine the at least one classification data set to obtain an abnormal classification data set, randomly select the first abnormal classification data, and construct a layer of classification nodes.

[0056] That is, at least one previously extracted classification data set is combined to form an abnormal classification data set, for example, the classification data of device 37 and device 15 are integrated into a set containing 300 data items.

[0057] A piece of data is randomly selected from the abnormal classification data set as the first abnormal classification data, and a layer of classification nodes is constructed based on this: this node performs binary classification on the input data. For example, if the first abnormal classification data is temperature 35℃, the layer node may classify the data of "temperature ≥ 35℃" into one category and "temperature < 35℃" into another category. When a piece of data becomes "single data" after classification, that is, it cannot be classified into the same category as other data, it is directly marked as abnormal data.

[0058] Based on the classification results of the first-layer classification nodes, the process of randomly selecting benchmark data and constructing the next-layer classification nodes is repeated for each type of data: for example, for the classification data set with temperature ≥35℃ in the first-layer node, a temperature data of 40℃ is randomly selected as the benchmark to construct the second-layer classification nodes, and this type of data is further divided into two categories: temperature ≥40℃ and temperature <40℃. Similarly, the isolated single data after classification is marked as abnormal.

[0059] Repeat the above process until the preset maximum number of layers is reached to form an anomaly classifier containing multiple layers of classification nodes. The more layers there are, the finer the classification. For example, the preset maximum number of layers can be 5 layers.

[0060] Then, each second data of the second data set is input into the anomaly classifier, and its classification path in each layer node is traced. If a data is judged as a single data, that is, anomaly data, at the third layer node, the number of node layers in which it is classified as anomaly data is 3.

[0061] Calculate the ratio of the number of node layers classified as abnormal to the maximum number of node layers as the credibility factor of the data. For example, if the maximum number of node layers is 5, the credibility factor of the second data is 3 / 5 = 0.6. A higher ratio indicates that the data was classified as abnormal after more layers of classification, and its credibility is higher, that is, it is more likely to be normal data; otherwise, the credibility is lower.

[0062] Finally, the credibility factors of all second data are calculated to form a second data credibility factor set.

[0063] The construction of the above classification nodes can be achieved through a multi-layer binary tree, and the nodes in each layer of the multi-layer binary tree are used to classify the data into two categories. The specific method of using the multi-layer binary tree to classify data is an existing technology and will not be repeated here.

[0064] In step S400 of the embodiment of the present application, based on the second data value factor set and the second data credibility factor set, a second data screening factor set is calculated, data greater than or equal to the data screening factor threshold is screened and added to the cumulative data set, and the cumulative data set of all entities is further screened to obtain as the data asset inventory result, including: Calculating a second data screening factor set based on the second data value factor set and the second data credibility factor set; Filter and obtain second data whose second data screening factor is greater than or equal to the data screening factor threshold, and add the second data to the accumulated data set; Continue to collect and filter data from other entities to obtain the cumulative data set of all entities as the result of the data asset inventory.

[0065] The purpose of step S400 in the embodiment of the present application is to achieve accurate screening and comprehensive integration of data assets, ensuring that the final inventory results contain both high-value and highly reliable data.

[0066] The logic behind this approach is to comprehensively evaluate each piece of secondary data from both the "value" and "credibility" dimensions, combining a value factor reflecting the data's uniqueness and usefulness with a credibility factor reflecting its reliability and normality, thus avoiding bias caused by single-dimensional screening. Furthermore, by screening secondary data that meets the data screening factor threshold, the cumulative dataset is updated and gradually covered across all entities. The resulting cumulative dataset fully and accurately represents the data assets of all entities, providing scientific inventory results for data asset management and application.

[0067] First, a second data screening factor set needs to be calculated based on the second data value factor set and the second data credibility factor set. That is, for each piece of second data in the second data set, the screening factor is calculated using a preset formula in combination with its corresponding second data value factor and second data credibility factor.

[0068] For example, the preset formula calculation can be in the form of weighted summation: second data screening factor = second data value factor × K1 + second data credibility factor × K2, and the weight can be adjusted according to the degree of reliance on data value and credibility. For example, K1=K2=0.5 can be set.

[0069] Integrate the screening factors of all second data to form a second data screening factor set, and complete the comprehensive quantitative evaluation of each data.

[0070] For example, if the value factor of a piece of second data of the device 15 is 0.8 and the credibility factor is 0.9, then the screening factor of the second data = 0.8×0.5+0.9×0.5=0.85.

[0071] Then, it is necessary to set a data screening factor threshold, which can be set according to the requirements for the data quality of the accumulated data set, such as 0.6.

[0072] Next, second data with a screening factor greater than or equal to the data screening factor threshold is screened out from the second data screening factor set, and these data are added to the cumulative data set, so that the cumulative data set is dynamically updated and includes high-quality data of the first subject and the second subject.

[0073] For example, if the data screening factor threshold is 0.6, the second data with the second data screening factor of 0.85 is retained and added to the cumulative data set; while the second data with the second data screening factor of 0.5 is removed.

[0074] Finally, according to the above process, the data of all remaining entities, such as the third entity and the fourth entity, are processed in turn.

[0075] When the data of all entities are collected and screened, the final cumulative data set is the data asset inventory result, which covers the data of all entities that are both high-value and highly reliable.

[0076] Example 2, as Figure 2 As shown, based on the same inventive concept as the data asset inventory method provided in the first embodiment, the embodiment of the present invention further provides a data asset inventory system, including: The cumulative data collection module 11 is configured to randomly select a first subject from a plurality of subjects and collect a first data set of the first subject as a cumulative data set, wherein the cumulative data set is updated as the subject data is traversed; A value factor analysis module 12 is configured to continue collecting a second data set of a second subject and analyze a second data value factor set of the second data set based on the accumulated data set; a data credibility analysis module 13 configured to configure at least one classification data extraction coefficient based on the proportion of data of each subject in the accumulated data set, extract at least one classification data set, construct an anomaly classifier, perform anomaly classification on the plurality of second data, and obtain a second data credibility factor set; The cumulative data screening module 14 is used to calculate the second data screening factor set based on the second data value factor set and the second data credibility factor set, screen and obtain data that is greater than or equal to the data screening factor threshold, add it to the cumulative data set, and continue to screen and obtain the cumulative data set of all entities as the data asset inventory result.

[0077] Furthermore, the cumulative data collection module 11 includes the following execution steps: Obtain multiple entities for data asset inventory; A first subject among the multiple subjects is selected, and a first data set of the first subject is collected as an accumulated data set, wherein the accumulated data set is updated as the subject data is traversed and collected.

[0078] Furthermore, the value factor analysis module 12 includes the following execution steps: collecting a second data set of a second subject, wherein the second data set includes a plurality of second data; According to the accumulated data set, second data value factors of a plurality of second data in the second data set are analyzed to obtain a second data value factor set.

[0079] The step of analyzing the second data value factors of the plurality of second data items in the second data set according to the accumulated data set to obtain a second data value factor set includes: Calculating the similarity between each second data in the second data set and all data sets in the cumulative data set, and calculating the average to obtain a second data similarity set; A second data value factor set is calculated based on the second data similarity set.

[0080] Furthermore, the data credibility analysis module 13 includes the following execution steps: Calculating the proportion of data of each subject in the accumulated data set as at least one classification data extraction coefficient; Obtaining a preset data extraction quantity, and calculating at least one classification data extraction amount based on the at least one classification data extraction coefficient; randomly extracting data of at least one subject from the accumulated data set according to the at least one classification data extraction quantity to obtain at least one classification data set; An anomaly classifier is constructed using the at least one classification data set, and an anomaly classification is performed on the second data to obtain a second data credibility factor set.

[0081] The at least one classification data set is used to construct an anomaly classifier, and anomaly classification is performed on the plurality of second data to obtain a second data credibility factor set, including: Combining the at least one classification data set to obtain an abnormal classification data set, randomly selecting first abnormal classification data, and constructing a layer of classification nodes, wherein the layer of classification nodes performs binary classification on the input data and marks the data as abnormal data when the data is classified as a single data; Continue to build and obtain multi-layer classification nodes to obtain anomaly classifiers; Input multiple second data into the anomaly classifier, obtain the node layer number of each second data being classified as anomaly data, calculate the ratio to the maximum node layer number, obtain multiple second data credibility factors, and integrate them to obtain a second data credibility factor set.

[0082] Furthermore, the cumulative data screening module 14 includes the following execution steps: Calculating a second data screening factor set based on the second data value factor set and the second data credibility factor set; Filter and obtain second data whose second data screening factor is greater than or equal to the data screening factor threshold, and add the second data to the accumulated data set; Continue to collect and filter data from other entities to obtain the cumulative data set of all entities as the result of the data asset inventory.

[0083] Example 3, as Figure 3As shown, based on the same inventive concept as the data asset-based inventory method provided in Example 1, an embodiment of the present invention further provides a storage medium 400. Exemplarily, the storage medium can be a non-transitory computer-readable storage medium, and a second computer software program 410 is stored in the storage medium. When the second computer software program 410 is executed by the processor, an inventory method based on data assets as in Example 1 is implemented.

[0084] Non-transitory storage media refers to storage media that can persist data after power failure, including but not limited to SSDs (solid-state drives), HDDs (hard disk drives), and flash memory devices (USB flash drives, memory cards).

[0085] It should be noted that, in the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0086] Those skilled in the art will appreciate that embodiments of the present invention may provide methods, systems, or computer program products. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0087] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0088] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0089] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0090] Although preferred embodiments of the present invention have been described, additional changes and modifications to these embodiments may occur to those skilled in the art once the basic inventive concepts become known.

[0091] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the present invention and its equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A data asset inventory method, characterized in that: The method comprises: Randomly selecting a first subject from a plurality of subjects, collecting a first data set of the first subject as an accumulated data set, wherein the accumulated data set is updated as the subject data is traversed; Continuing to collect a second data set of a second subject, and analyzing a second data value factor set of the second data set based on the accumulated data set; configuring at least one classification data extraction coefficient according to the proportion of data of each subject in the accumulated data set, extracting at least one classification data set, constructing an anomaly classifier, performing anomaly classification on the plurality of second data, and obtaining a second data credibility factor set; Based on the second data value factor set and the second data credibility factor set, a second data screening factor set is calculated, and data greater than or equal to the data screening factor threshold is screened and added to the cumulative data set. The cumulative data set of all entities is further screened as the data asset inventory result.

2. The data asset inventory method according to claim 1, characterized in that: A first subject is randomly selected from a plurality of subjects, and a first data set of the first subject is collected as a cumulative data set, including: Obtain multiple entities for data asset inventory; A first subject among the multiple subjects is selected, and a first data set of the first subject is collected as an accumulated data set, wherein the accumulated data set is updated as the subject data is traversed and collected.

3. The data asset inventory method according to claim 1, characterized in that: Continuing to collect a second data set of a second subject, and analyzing a second data value factor set of the second data set based on the accumulated data set, including: collecting a second data set of a second subject, wherein the second data set includes a plurality of second data; According to the accumulated data set, second data value factors of a plurality of second data in the second data set are analyzed to obtain a second data value factor set.

4. The data asset inventory method according to claim 3, characterized in that: Analyzing, according to the accumulated data set, second data value factors of a plurality of second data items in the second data set to obtain a second data value factor set includes: Calculating the similarity between each second data in the second data set and all data sets in the cumulative data set, and calculating the average to obtain a second data similarity set; A second data value factor set is calculated based on the second data similarity set.

5. The data asset inventory method according to claim 1, characterized in that: According to the proportion of data of each subject in the accumulated data set, at least one classification data extraction coefficient is configured, at least one classification data set is extracted, an anomaly classifier is constructed, and anomaly classification is performed on the plurality of second data to obtain a second data credibility factor set, including: Calculating the proportion of data of each subject in the accumulated data set as at least one classification data extraction coefficient; Obtaining a preset data extraction quantity, and calculating at least one classification data extraction amount based on the at least one classification data extraction coefficient; randomly extracting data of at least one subject from the accumulated data set according to the at least one classification data extraction quantity to obtain at least one classification data set; An anomaly classifier is constructed using the at least one classification data set, and an anomaly classification is performed on the second data to obtain a second data credibility factor set.

6. The data asset inventory method according to claim 5, characterized in that: Using the at least one classification data set to construct an anomaly classifier, performing anomaly classification on the plurality of second data, and obtaining a second data credibility factor set, including: Combining the at least one classification data set to obtain an abnormal classification data set, randomly selecting first abnormal classification data, and constructing a layer of classification nodes, wherein the layer of classification nodes performs binary classification on the input data and marks the data as abnormal data when the data is classified as a single data; Continue to build and obtain multi-layer classification nodes to obtain anomaly classifiers; Input multiple second data into the anomaly classifier, obtain the node layer number of each second data being classified as anomaly data, calculate the ratio to the maximum node layer number, obtain multiple second data credibility factors, and integrate them to obtain a second data credibility factor set.

7. The data asset inventory method according to claim 1, characterized in that: Based on the second data value factor set and the second data credibility factor set, a second data screening factor set is calculated, data greater than or equal to the data screening factor threshold is screened and added to the cumulative data set, and the cumulative data set of all entities is continuously screened to obtain as the data asset inventory result, including: Calculating a second data screening factor set based on the second data value factor set and the second data credibility factor set; Filter and obtain second data whose second data screening factor is greater than or equal to the data screening factor threshold, and add the second data to the accumulated data set; Continue to collect and filter data from other entities to obtain the cumulative data set of all entities as the result of the data asset inventory.

8. An inventory system based on data assets, characterized in that: The system comprises: an accumulated data collection module, configured to randomly select a first subject from a plurality of subjects, collect a first data set of the first subject as an accumulated data set, wherein the accumulated data set is updated as the data of the subjects are traversed; a value factor analysis module, configured to continue collecting a second data set of a second subject, and analyze a second data value factor set of the second data set based on the accumulated data set; a data credibility analysis module, configured to configure at least one classification data extraction coefficient based on the proportion of data of each subject in the accumulated data set, extract at least one classification data set, construct an anomaly classifier, perform anomaly classification on the plurality of second data, and obtain a second data credibility factor set; The cumulative data screening module is used to calculate the second data screening factor set based on the second data value factor set and the second data credibility factor set, screen and obtain data greater than or equal to the data screening factor threshold, add it to the cumulative data set, and continue to screen and obtain the cumulative data set of all entities as the data asset inventory result.

9. A storage medium, characterized in that: The storage medium stores a second computer software program, which, when executed by the processor, implements the data asset-based inventory method according to any one of claims 1 to 7.

Images