Unstructured data management method and system

Abstract

The invention relates to the technical field of data management and storage, in particular to an unstructured data management method and system. The method comprises the following steps: acquiring unstructured data in an industrial environment, and recording acquisition time; setting a sliding time window and completely storing the unstructured data falling into the window; when an error event occurs in the industrial environment, acquiring occurrence time of the error event; for each piece of unstructured data, calculating a time difference between the acquisition time of the piece of unstructured data and the occurrence time of the error event, and substituting the time difference into a preset unstructured data weight calculation function to obtain a storage weight of the piece of unstructured data; when each piece of unstructured data is discharged out of the preset sliding time window, the storage quality of the piece of unstructured data is adjusted according to the final storage weight of the piece of unstructured data, so that the piece of unstructured data is finally stored according to the storage quality. According to the method provided by the invention, the storage efficiency of the industrial unstructured data can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of unstructured data management and storage, and in particular to an unstructured data management method and system. Background Art

[0002] In current industrial production, a large amount of unstructured data is generated in equipment operation, production process, quality inspection and other links, such as equipment logs, fault records, quality inspection reports, etc. These unstructured data contain valuable information and are of great value for equipment health management, production process optimization, product quality improvement, etc.

[0003] The generation speed of unstructured data far exceeds that of structured data, and it appears in multiple formats such as text, images, audio and video, which places high demands on the capacity and processing power of the storage system.

[0004] Traditional data storage solutions are unable to effectively cope with the above characteristics and challenges of unstructured industrial data. Faced with massive and multi-type unstructured data, traditional solutions often adopt a "one-size-fits-all" storage strategy, ignoring the value differences of different data, resulting in a huge waste of storage resources. Summary of the invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides an unstructured data management method and system, which can improve the efficiency of industrial unstructured data storage.

[0006] In a first aspect, the present application provides an unstructured data management method, the unstructured data management method comprising the following steps: Acquire unstructured data in industrial environments and record the acquisition time of each unstructured data; Setting a sliding time window, and completely storing the unstructured data falling within the preset sliding time window; When an error event occurs in an industrial environment, obtain the time when the error event occurs; For each piece of unstructured data, calculate the time difference between its acquisition time and the time when the error event occurred, substitute the time difference into the preset unstructured data weight calculation function, and obtain the storage weight of the unstructured data; When each piece of unstructured data is discharged from the preset sliding time window, the storage quality of the unstructured data is adjusted according to its final storage weight so that it is finally stored according to the storage quality.

[0007] Optionally, the preset unstructured data weight calculation function also includes: Obtain the type of production event based on the error events that have occurred within the sliding time window; According to the type of production event, the correlation strength of each type of unstructured data is calculated through the preset industrial expert knowledge decision tree; The unstructured data weight calculation function also calculates the storage weight of each piece of unstructured data according to the correlation strength of each type of unstructured data.

[0008] Optionally, for unstructured data of data type k, the storage weight of unstructured data i collected at time point t is calculated by the following unstructured data weight calculation function: Among them, R(k) is the correlation strength of unstructured data of data type k, is the severity coefficient of the jth error event, is the occurrence time of the jth error event, n is the total number of error events in the sliding time window, and is an empirical parameter.

[0009] Optionally, the industrial expert knowledge decision tree is constructed by the following steps: Experts summarize the experience of associating production events with unstructured data in industrial production practice; A decision tree is constructed, in which each root node represents a type of production event, each non-leaf node represents a judgment condition for branching judgment on the type of unstructured data, and each leaf node represents the strength of association, indicating the strength of association between the production event on the path and the type of unstructured data.

[0010] Optionally, the length of the sliding time window is determined by the following steps: According to the preset industrial environment production observation indicators, obtain the production load of the current industrial environment; According to the production load of the current industrial environment and the preset maximum load threshold, the length of the sliding time window is calculated by a preset sliding time window calculation function; The setting rule of the sliding time window calculation function is such that the closer the production load of the current industrial environment is to the preset maximum load threshold, the shorter the length of the sliding time window is; According to the error events that have occurred in the sliding time window, obtaining the type of production events includes the following steps: Obtain each error event within the sliding time window and establish an error event correlation feature vector; The associated feature vector is input into a preset event classification model to obtain the production event type within the sliding time window.

[0011] Optionally, the preset sliding time window calculation function is: Among them, L(t) is the length of the sliding time window at time t, is the preset maximum window length, P(t) is the production load at time t, is the normal production load benchmark value, is the preset maximum load threshold, and is an empirical parameter.

[0012] Optionally, adjusting the storage quality of the unstructured data according to its final storage weight includes the following steps: Based on a preset storage weight threshold interval, a plurality of discrete storage levels are divided, each storage level corresponds to a predefined storage quality combination, and the storage quality combination includes one or more storage quality parameters of storage medium type, data compression ratio, number of copies and encryption strength; Compare the final storage weight of each piece of unstructured data with the storage weight threshold interval to determine the storage level to which it belongs, and obtain the storage quality combination corresponding to the storage level; According to each storage quality parameter included in the storage quality combination, each piece of unstructured data is processed accordingly; The processed unstructured data is stored in the corresponding storage medium according to the requirements of its storage quality combination.

[0013] In a second aspect, the present application provides an unstructured data management system, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement an unstructured data management method as described in any one of the first aspects.

[0014] Compared with the prior art, the technical solution provided by this application has the following advantages: One of its beneficial effects is that the present application proposes an unstructured data management method in an industrial environment, aiming to effectively address the problems of rapid growth of unstructured data, high data duplication, and uneven data value in the industrial production process, and to achieve optimized storage and efficient use of data.

[0015] First, this application obtains unstructured data generated in the industrial environment, such as equipment operating parameters, product quality information, audio and video monitoring data, etc., and records the acquisition time of each piece of data.

[0016] Then, this application sets a predefined sliding time window and stores all unstructured data falling within the window in full. The purpose is to temporarily collect and save data in the recent period with high integrity, laying the foundation for subsequent data analysis.

[0017] When an error event occurs in the industrial production process (such as a machine downtime, the temperature of the production environment exceeds the warning value, etc.), this application will automatically capture the time point of the event. Next, for each piece of unstructured data, this application will calculate the difference between its collection time and the time when the error event occurred, and substitute the difference into a preset data weight calculation function.

[0018] This weight calculation function is a key point of this application. It comprehensively considers the temporal correlation between the data and the error event and obtains a quantitative weight value. The smaller the time difference of the data, the higher its weight value, because the closer they are to the time of the error, the more targeted and diagnostically valuable the information they contain.

[0019] When each piece of unstructured data has passed through the sliding time window and leaves the window boundary, this application will dynamically adjust its storage quality. This adjustment is based on the final weight value of the data: data with high weight will be stored in storage media with faster access speeds (such as SSD) and use a lower compression ratio to ensure fast data reading and analysis; while data with low weight will be transferred to storage media with slower access speeds (such as HDD) and use a higher compression ratio to save storage space.

[0020] One of the beneficial effects of this application is mainly reflected in the following aspects: 1. Improve the collection and storage ratio of key data. Through the sliding window mechanism and data weight calculation, this application can automatically identify data with high value for error analysis, cause diagnosis, etc., and give priority and focused storage, thereby improving the capture rate of key data and reducing the risk of data omission.

[0021] 2. Optimize the configuration of storage resources. Traditional unstructured data management often adopts a "one-size-fits-all" storage strategy, ignoring the value differences of different data and causing a waste of storage resources. However, this application dynamically evaluates the weight of data, prioritizes storage and quick access to high-value data, and compresses and stores low-value data or transfers it to low-speed media, thereby achieving optimized configuration and efficient use of storage resources.

[0022] 3. Reduce the overall cost of data storage. By hierarchically managing and dynamically optimizing data storage, this application not only improves storage performance, but also saves storage space and storage costs. Especially for scenarios with massive unstructured data, the cost reduction and efficiency improvement are particularly obvious.

[0023] To sum up, the unstructured data management method in an industrial environment proposed in this application realizes the "on-demand allocation and dynamic optimization" of data storage, and has achieved remarkable technical effects in improving the key data collection rate, accelerating data analysis and diagnosis, and reducing the total storage cost.

[0024] The second beneficial effect is that it should be pointed out that a collection of different error events indicates different production events (such as equipment failure events, product quality events, and efficiency reduction events, etc.). In different production events, the relevance of different types of unstructured data to production events is also different, so all types of unstructured data cannot be stored equally.

[0025] This application further introduces a data value assessment mechanism based on industrial expert knowledge. This mechanism quantifies the correlation between different types of production events and unstructured data by leveraging the experience of industrial experts, thereby further achieving more accurate judgment of data value and more targeted storage optimization.

[0026] The third beneficial effect is that the present application provides a method for calculating a sliding time window, which can make the length of the sliding time window shorter when the production load is greater, thereby adapting to different occurrence logics of error events under different load conditions.

[0027] When working under high load, the equipment is in a critical state, and error events occur suddenly and in a concentrated manner, that is, a large number of error events occur in a short period of time and there is a strong correlation between the error events.

[0028] By shortening the length of the sliding time window when working under high load.

[0029] On the one hand, the unstructured data weight calculation function provided by the present application is used to calculate the storage weight of unstructured data. The range of the value of becomes smaller, making As the overall size increases, T will also increase. When an error event occurs in the sliding time window, unstructured data can be stored with a higher storage weight.

[0030] On the other hand, since error events change rapidly under high load conditions, shortening the sliding time window improves the correlation of error events within the sliding time window, which helps the event classification model to accurately identify typical failure modes, thereby more accurately obtaining the production event type within the sliding time window, and more accurately determining which unstructured data should be stored.

[0031] By combining the adjustments of the two parts, the present application can more accurately save the relevant unstructured data with high quality by shortening the time window under high load conditions.

[0032] However, when working at low load, the equipment is in a state of maintenance, servicing or low-speed operation, and the occurrence of error events is sparse, that is, a large number of errors will not occur in a short period of time.

[0033] When working under low load, the length of the sliding time window is lengthened.

[0034] On the one hand, using the unstructured data weight calculation function provided by this application, although the longer time window makes The overall value becomes smaller, but by lengthening the sliding time window, it can accumulate as many error events as possible. When T still encounters the abnormal phenomenon of multiple error events in a sliding time window in the case of such occasional error events, it can be increased to capture this abnormal phenomenon, so that the unstructured data during this abnormal time period can be stored as key data with higher quality. This adapts to the scattered characteristics of error events under low load conditions, and accurately identifies the unstructured data that really needs high-quality storage through longer data accumulation.

[0035] On the other hand, since error events change slowly and in small numbers under low load conditions, by lengthening the sliding time window and achieving long-term data accumulation, the event classification model can identify slowly changing failure modes, thereby more accurately obtaining the production event type within the sliding time window.

[0036] By combining the adjustments of the two parts, the present application can more accurately preserve the relevant unstructured data with high quality by extending the time window under low load conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A flowchart of an unstructured data management method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0039] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application can also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only part of the embodiments of the present application, not all of the embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0040] In a first aspect, the present application provides an unstructured data management method, the unstructured data management method comprising the following steps: S101: Acquire unstructured data in an industrial environment, and record the acquisition time of each unstructured data.

[0041] Specifically, in this embodiment, the unstructured data in the industrial environment mainly includes: equipment operating parameters (such as temperature, pressure and vibration, etc.), quality detection data (such as images and videos, etc.), environmental monitoring data (such as noise and air data, etc.), and operation log data, etc. These data can be automatically collected through various sensors, detection equipment, monitoring systems, etc., or obtained through manual entry. For each piece of acquired data, the system will automatically associate a timestamp to record the exact time when it was generated or collected.

[0042] S102: Setting a sliding time window, and completely storing the unstructured data falling within the preset sliding time window.

[0043] Specifically, the length of the sliding time window is determined by the following steps: According to the preset industrial environment production observation indicators, the production load of the current industrial environment is obtained.

[0044] In an embodiment of the present application, production observation indicators in an industrial environment are obtained through a production management system. These indicators include equipment utilization, material consumption rate and energy consumption. These indicators are weighted and summed to obtain the production load of the current industrial environment, where the weight used is a value set artificially based on experience.

[0045] According to the production load of the current industrial environment and the preset maximum load threshold, the length of the sliding time window is calculated by a preset sliding time window calculation function; The setting rule of the sliding time window calculation function is such that the closer the production load of the current industrial environment is to a preset maximum load threshold, the shorter the length of the sliding time window is.

[0046] Specifically, in the embodiment of the present application, the preset sliding time window calculation function is: Among them, L(t) is the length of the sliding time window at time t, is the preset maximum window length, P(t) is the production load at time t, It is a normal production load reference value preset by humans. It is the maximum load threshold preset by humans. and It is an artificially preset empirical parameter.

[0047] S103: When an error event occurs in the industrial environment, obtain the occurrence time of the error event; For each piece of unstructured data, the time difference between its acquisition time and the time when the error event occurred is calculated, and the time difference is substituted into the preset unstructured data weight calculation function to obtain the storage weight of the unstructured data.

[0048] Specifically, in the embodiment of the present application, the error event is an error such as a machine shutting down, the temperature of the production environment exceeding the temperature warning value, the number of substandard products exceeding the predetermined warning value, etc. In other embodiments, the error event can be set. Generally speaking, when the industrial indicator data exceeds the normal threshold, it can be determined that an error event has occurred.

[0049] Therefore, in the embodiment of the present application, when judging whether an error event occurs in an industrial environment, it is possible to judge whether the industrial indicator data is abnormal automatically or by manual entry.

[0050] Specifically, in the embodiment of the present application, the preset unstructured data weight calculation function also includes: Obtain the type of production event based on the error events that have occurred within the sliding time window; Specifically, according to the error events that have occurred in the sliding time window, obtaining the type of the production event includes the following steps: Obtain each error event within the sliding time window and establish an error event correlation feature vector; The associated feature vector is input into a preset event classification model to obtain the production event type within the sliding time window.

[0051] Specifically, the event classification model is trained through the following steps: In fact, in the embodiment of the present application, the type of production event is the type of error event, for example, equipment downtime corresponds to equipment failure, substandard products exceeding a predetermined warning value correspond to quality defects, and the temperature of the production environment exceeding a temperature warning value corresponds to a safety accident. The purpose of setting production events is to describe the types of mainstream error events within a period of time.

[0052] In the embodiment of the present application, the event classification model is actually a judgment machine, which judges whether the production event is of a certain type by detecting whether the number of error events of a certain type in a sliding time window exceeds a preset judgment threshold.

[0053] In other embodiments, the basic network framework of the event classification model can be set as a fully connected neural network, and the error event sets occurring in multiple sliding time windows and their corresponding production events are collected. The error event sets occurring in the sliding time windows are used as input, and the types of production events are used as labels, so as to train and obtain the event classification model.

[0054] According to the type of production event, the correlation strength of each type of unstructured data is calculated through the preset industrial expert knowledge decision tree; The unstructured data weight calculation function also calculates the storage weight of each piece of unstructured data according to the correlation strength of each type of unstructured data.

[0055] Specifically, the industrial expert knowledge decision tree is constructed by the following steps: Experts summarize the experience of associating production events with different types of unstructured data in industrial production practice.

[0056] A decision tree is constructed, in which each root node represents a type of production event, each non-leaf node represents a judgment condition for branching judgment on the type of unstructured data, and each leaf node represents the strength of association, indicating the strength of association between the production event on the path and the type of unstructured data.

[0057] Specifically, for unstructured data of data type k, the storage weight of unstructured data i collected at time point t is calculated by the following unstructured data weight calculation function: Among them, R(k) is the correlation strength of unstructured data of data type k, is the severity coefficient of the jth error event, is the occurrence time of the jth error event, n is the total number of error events in the sliding time window, and It is an artificially set empirical parameter.

[0058] The severity coefficient of the error event is obtained by table lookup in the embodiment of the present application, that is, multiple error event types and their corresponding severity coefficients are artificially formulated in advance, and when needed, the severity coefficient is directly obtained by table lookup based on the current error event type.

[0059] S104: When each piece of unstructured data is excluded from the preset sliding time window, the storage quality of the unstructured data is adjusted according to its final storage weight, so that it is finally stored according to the storage quality.

[0060] Specifically, adjusting the storage quality of the unstructured data according to its final storage weight includes the following steps: Based on the preset storage weight threshold range, multiple discrete storage levels are divided.

[0061] That is, multiple preset storage weight threshold intervals are set, and each interval corresponds to a storage level.

[0062] Each storage level corresponds to a predefined storage quality combination, wherein the storage quality combination includes one or more storage quality parameters of storage medium type, data compression ratio, number of copies, and encryption strength; Compare the final storage weight of each piece of unstructured data with the storage weight threshold interval to determine the storage level to which it belongs, and obtain the storage quality combination corresponding to the storage level; According to each storage quality parameter included in the storage quality combination, each piece of unstructured data is processed accordingly; The processed unstructured data is stored in a database consisting of corresponding storage media according to the requirements of its storage quality combination.

[0063] Compared with the prior art, the technical solution provided by this application has the following advantages: One of its beneficial effects is that the present application proposes an unstructured data management method in an industrial environment, aiming to effectively address the problems of rapid growth of unstructured data, high data duplication, and uneven data value in the industrial production process, and to achieve optimized storage and efficient use of data.

[0064] First, this application obtains unstructured data generated in the industrial environment, such as equipment operating parameters, product quality information, audio and video monitoring data, etc., and records the acquisition time of each piece of data.

[0065] Then, this application sets a predefined sliding time window and stores all unstructured data falling within the window in full. The purpose is to temporarily collect and save data in the recent period with high integrity, laying the foundation for subsequent data analysis.

[0066] When an error event occurs in the industrial production process (such as a machine downtime, the temperature of the production environment exceeds the warning value, etc.), this application will automatically capture the time point of the event. Next, for each piece of unstructured data, this application will calculate the difference between its collection time and the time when the error event occurred, and substitute the difference into a preset data weight calculation function.

[0067] This weight calculation function is a key point of this application. It comprehensively considers the temporal correlation between the data and the error event and obtains a quantitative weight value. The smaller the time difference of the data, the higher its weight value, because the closer they are to the time of the error, the more targeted and diagnostically valuable the information they contain.

[0068] When each piece of unstructured data has passed through the sliding time window and leaves the window boundary, this application will dynamically adjust its storage quality. This adjustment is based on the final weight value of the data: data with high weight will be stored in storage media with faster access speeds (such as SSD) and use a lower compression ratio to ensure fast data reading and analysis; while data with low weight will be transferred to storage media with slower access speeds (such as HDD) and use a higher compression ratio to save storage space.

[0069] One of the beneficial effects of this application is mainly reflected in the following aspects: 1. Improve the collection and storage ratio of key data. Through the sliding window mechanism and data weight calculation, this application can automatically identify data with high value for error analysis, cause diagnosis, etc., and give priority and focused storage, thereby improving the capture rate of key data and reducing the risk of data omission.

[0070] 2. Optimize the configuration of storage resources. Traditional unstructured data management often adopts a "one-size-fits-all" storage strategy, ignoring the value differences of different data and causing a waste of storage resources. However, this application dynamically evaluates the weight of data, prioritizes storage and quick access to high-value data, and compresses and stores low-value data or transfers it to low-speed media, thereby achieving optimized configuration and efficient use of storage resources.

[0071] 3. Reduce the overall cost of data storage. By hierarchically managing and dynamically optimizing data storage, this application not only improves storage performance, but also saves storage space and storage costs. Especially for scenarios with massive unstructured data, the cost reduction and efficiency improvement are particularly obvious.

[0072] To sum up, the unstructured data management method in an industrial environment proposed in this application realizes the "on-demand allocation and dynamic optimization" of data storage, and has achieved remarkable technical effects in improving the key data collection rate, accelerating data analysis and diagnosis, and reducing the total storage cost.

[0073] The second beneficial effect is that it should be pointed out that a collection of different error events indicates different production events (such as equipment failure events, product quality events, and efficiency reduction events, etc.). In different production events, the relevance of different types of unstructured data to production events is also different, so all types of unstructured data cannot be stored equally.

[0074] This application further introduces a data value assessment mechanism based on industrial expert knowledge. This mechanism quantifies the correlation between different types of production events and unstructured data by leveraging the experience of industrial experts, thereby further achieving more accurate judgment of data value and more targeted storage optimization.

[0075] The third beneficial effect is that the present application provides a method for calculating a sliding time window, which can make the length of the sliding time window shorter when the production load is greater, thereby adapting to different occurrence logics of error events under different load conditions.

[0076] When working under high load, the equipment is in a critical state, and error events occur suddenly and in a concentrated manner, that is, a large number of error events occur in a short period of time and there is a strong correlation between the error events.

[0077] By shortening the length of the sliding time window when working under high load.

[0078] On the one hand, the unstructured data weight calculation function provided by the present application is used to calculate the storage weight of unstructured data. The range of the value of becomes smaller, making As the overall size increases, T will also increase. When an error event occurs in the sliding time window, unstructured data can be stored with a higher storage weight.

[0079] On the other hand, since error events change rapidly under high load conditions, shortening the sliding time window improves the correlation of error events within the sliding time window, which helps the event classification model to accurately identify typical failure modes, thereby more accurately obtaining the production event type within the sliding time window, and more accurately determining which unstructured data should be stored.

[0080] By combining the adjustments of the two parts, the present application can more accurately save the relevant unstructured data with high quality by shortening the time window under high load conditions.

[0081] However, when working at low load, the equipment is in a state of maintenance, servicing or low-speed operation, and the occurrence of error events is sparse, that is, a large number of errors will not occur in a short period of time.

[0082] When working under low load, the length of the sliding time window is lengthened.

[0083] On the one hand, using the unstructured data weight calculation function provided by this application, although the longer time window makes The overall value becomes smaller, but by lengthening the sliding time window, it can accumulate as many error events as possible. When T still encounters multiple error events in a sliding time window in the case of such an occasional error event, it can be increased to capture this abnormal phenomenon, so that the unstructured data during this abnormal time can be stored with higher quality. This adapts to the scattered characteristics of error events under low load conditions, and accurately identifies the unstructured data that really needs high-quality storage through longer data accumulation.

[0084] On the other hand, since error events change slowly and in small numbers under low load conditions, by lengthening the sliding time window and achieving long-term data accumulation, the event classification model can identify slowly changing failure modes, thereby more accurately obtaining the production event type within the sliding time window.

[0085] By combining the adjustments of the two parts, the present application can more accurately preserve the relevant unstructured data with high quality by extending the time window under low load conditions.

[0086] In a second aspect, the present application provides an unstructured data management system, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the unstructured data management method as described in any of the above embodiments.

[0087] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. In addition, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. Moreover, in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can represent A or B; "and / or" in this article is only a kind of association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. Furthermore, in the description of the embodiments of the present application, “plurality” refers to two or more than two.

[0088] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An unstructured data management method, characterized in that: The unstructured data management method comprises the following steps: Acquire unstructured data in industrial environments and record the acquisition time of each unstructured data; Setting a sliding time window, and completely storing the unstructured data falling within the preset sliding time window; When an error event occurs in an industrial environment, obtain the time when the error event occurs; For each piece of unstructured data, calculate the time difference between its acquisition time and the time when the error event occurred, substitute the time difference into the preset unstructured data weight calculation function, and obtain the storage weight of the unstructured data; When each piece of unstructured data is discharged from the preset sliding time window, the storage quality of the unstructured data is adjusted according to its final storage weight so that it is finally stored according to the storage quality.

2. The unstructured data management method according to claim 1, characterized in that: The preset unstructured data weight calculation function also includes: Obtain the type of production event based on the error events that have occurred within the sliding time window; According to the type of production event, the correlation strength of each type of unstructured data is calculated through the preset industrial expert knowledge decision tree; The unstructured data weight calculation function also calculates the storage weight of each piece of unstructured data according to the correlation strength of each type of unstructured data.

3. The unstructured data management method according to claim 2, characterized in that: For unstructured data of data type k, the storage weight of unstructured data i collected at time point t is calculated by the following unstructured data weight calculation function: Among them, R(k) is the correlation strength of unstructured data of data type k, is the severity coefficient of the jth error event, is the occurrence time of the jth error event, n is the total number of error events in the sliding time window, and is an empirical parameter.

4. The unstructured data management method according to claim 2, characterized in that: The industrial expert knowledge decision tree is constructed by the following steps: Experts summarize the experience of associating production events with unstructured data in industrial production practice; A decision tree is constructed, in which each root node represents a type of production event, each non-leaf node represents a judgment condition for branching judgment on the type of unstructured data, and each leaf node represents the strength of association, indicating the strength of association between the production event on the path and the type of unstructured data.

5. The unstructured data management method according to claim 2, characterized in that: The length of the sliding time window is determined by the following steps: According to the preset industrial environment production observation indicators, obtain the production load of the current industrial environment; According to the production load of the current industrial environment and the preset maximum load threshold, the length of the sliding time window is calculated by a preset sliding time window calculation function; The setting rule of the sliding time window calculation function is such that the closer the production load of the current industrial environment is to the preset maximum load threshold, the shorter the length of the sliding time window is; According to the error events that have occurred in the sliding time window, obtaining the type of production events includes the following steps: Obtain each error event within the sliding time window and establish an error event correlation feature vector; The associated feature vector is input into a preset event classification model to obtain the production event type within the sliding time window.

6. The unstructured data management method according to claim 5, characterized in that: The preset sliding time window calculation function is: Among them, L(t) is the length of the sliding time window at time t, is the preset maximum window length, P(t) is the production load at time t, is the normal production load benchmark value, is the preset maximum load threshold, and is an empirical parameter.

7. The unstructured data management method according to claim 1, characterized in that: According to the final storage weight, the storage quality of the unstructured data is adjusted, including the following steps: Based on a preset storage weight threshold interval, a plurality of discrete storage levels are divided, each storage level corresponds to a predefined storage quality combination, and the storage quality combination includes one or more storage quality parameters of storage medium type, data compression ratio, number of copies and encryption strength; Compare the final storage weight of each piece of unstructured data with the storage weight threshold interval to determine the storage level to which it belongs, and obtain the storage quality combination corresponding to the storage level; According to each storage quality parameter included in the storage quality combination, each piece of unstructured data is processed accordingly; The processed unstructured data is stored in the corresponding storage medium according to the requirements of its storage quality combination.

8. Unstructured data management system, characterized in that, It includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the unstructured data management method as described in any one of claims 1-7.

Images