Data enhancement method, device, electronic device and storage medium
By constructing a directed graph of state knowledge and switching data based on directed edges, the generated target enhancement data is more in line with real scenarios, solving the problem of data enhancement methods lacking high-level features and overfitting in the existing technology, and improving the data Enhanced accuracy and generalization capabilities of the model.
Patent Information
- Application Number
- CN202411245632.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-06
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-09-06
AI Technical Summary
When generating additional sample data, existing data augmentation methods lack high-level features, the generative model training process is complex and prone to overfitting, which reduces the sample accuracy obtained by data augmentation.
By obtaining the historical operation data of the target device, building a directed graph of state knowledge, switching data based on directed edges and attribute configuration information, generating target enhancement data that is more suitable for real scenes, avoiding overfitting, and improving the accuracy of data enhancement sex.
It improves the accuracy of the additional sample data obtained by data augmentation, reduces the dependence on a large amount of training data, and enhances the interpretability and generalization ability of the model.
Smart Images

Figure CN118780397B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of machine learning technology, and in particular to a data enhancement method, device, electronic device and storage medium. Background Art
[0002] Data augmentation is a technique for solving data imbalance in machine learning and deep learning. It generates new samples that appear different but actually maintain the characteristics of the original data by performing various transformations and operations on the existing data set, thereby expanding the size and diversity of the data set. The purpose of data augmentation is to reduce the risk of model overfitting and improve the model's ability to generalize to new data without adding additional real data.
[0003] In related technologies, data augmentation is usually achieved through basic transformations or generative models. However, the data generated by simple basic transformations lacks higher-level features, and the training process of generative models is complex and prone to overfitting, which reduces the accuracy of additional sample data obtained by data augmentation. Summary of the invention
[0004] The embodiments of the present application provide a data enhancement method, device, electronic device and storage medium, which can generate samples that are more in line with real scenes and improve the accuracy of target enhanced data obtained through data enhancement.
[0005] To achieve the above-mentioned purpose, the first aspect of an embodiment of the present application proposes a data enhancement method, which includes: obtaining historical operation data of a target device, wherein the historical operation data includes the operation records of the target device in the past period of time and the expert experience data and industry rules of the target device in various application scenarios; constructing a state knowledge directed graph based on the historical operation data, wherein the state knowledge directed graph includes multiple state nodes, directed edges between the state nodes, and attribute configuration information on each directed edge; obtaining initial data, and inputting the initial data into the state knowledge directed graph so that the initial data switches states between multiple state nodes based on the directed edges; based on each directed edge and its corresponding attribute configuration information, obtaining the target data corresponding to the initial data when switching to each state node; when the data volume of the target data meets the preset termination condition, integrating all the target data to obtain target enhanced data.
[0006] In some embodiments, constructing a state knowledge directed graph based on the historical operation data includes: determining the independent state of the target device in the operation cycle based on the historical operation data; establishing multiple state nodes based on the operation cycle and the independent state, wherein each state node represents one of the independent states in the operation cycle; determining the node attributes under the state node based on the historical operation data, wherein the node attributes include the node name, basic attributes and data transformation method of the state node.
[0007] In some embodiments, after determining the node attributes under the state node according to the historical operation data, it also includes: determining the transition relationship between the state nodes and the transition direction of the transition relationship based on the historical operation data; establishing the directed edges between the state nodes according to the transition relationship between the state nodes and the transition direction of the transition relationship, wherein the direction of the directed edges represents the transition direction; determining the transition probability and transition condition between the state node and the directed edge according to the historical operation data; and generating the attribute configuration information on each of the directed edges according to the transition probability and the transition condition.
[0008] In some embodiments, the obtaining of initial data and inputting the initial data into the state knowledge directed graph so that the initial data switches states between a plurality of the state nodes based on the directed edges include: determining an initial state node among the state nodes, obtaining initial data and inputting the initial data into the state knowledge directed graph so that the initial data switches states based on the initial state node; determining a next state node to which the initial state node switches states according to the directed edges and the attribute configuration information of the initial state node so that the initial data switches from the initial state node to the next state node.
[0009] In some embodiments, when the data volume of the target data satisfies a preset termination condition, all the target data are integrated to obtain target enhanced data, including: when the initial data traversal switches to all state nodes in the state knowledge directed graph, all the target data are integrated to obtain target enhanced data; or, a preset target data threshold is obtained, and when the total amount of all the target data is greater than or equal to the target data threshold, all the target data are integrated to obtain target enhanced data.
[0010] In some embodiments, the method also includes: generating a fault node based on the historical operation data, wherein the fault node characterizes the fault and abnormal scenarios of the target device; adding the fault node to the state knowledge directed graph, generating a directed edge between the fault node and the state node, and updating the state knowledge directed graph so that the initial data is switched to the fault node based on the directed edge to generate target data of a negative sample.
[0011] In some embodiments, the method further includes: obtaining feedback information of the target enhancement data during model training and application, wherein the feedback information includes the prediction accuracy on the target enhancement data during model training; adjusting the node attributes of the state nodes in the state knowledge directed graph, the transition probability and transition conditions between the directed edges, the fault nodes or the preset termination conditions according to the feedback information.
[0012] To achieve the above-mentioned purpose, the second aspect of an embodiment of the present application proposes a data enhancement device, including: a historical data acquisition module, used to acquire the historical operation data of the target device; a state knowledge directed graph construction module, used to construct a state knowledge directed graph based on the historical operation data, wherein the state knowledge directed graph includes multiple state nodes, directed edges between the state nodes, and attribute configuration information on each directed edge; a state switching module, used to acquire initial data, input the initial data into the state knowledge directed graph, so that the initial data switches between the multiple state nodes based on the directed edges; a target data module, used to obtain the target data corresponding to the initial data when switching to each state node based on each directed edge and its corresponding attribute configuration information; a target enhancement data module, used to integrate all the target data to obtain target enhanced data when the data volume of the target data meets the preset termination condition.
[0013] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the data enhancement method described in the first aspect when executing the computer program.
[0014] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program. When the computer program is executed by a processor, it implements the data enhancement method described in the first aspect.
[0015] The embodiments of the present application propose a data enhancement method, device, electronic device and storage medium, which obtain historical operation data of a target device; construct a state knowledge directed graph based on the historical operation data, wherein the historical operation data includes the operation records of the target device in the past period of time and the expert experience data and industry rules of the target device in various application scenarios, and the state knowledge directed graph is a knowledge graph including multiple state nodes, directed edges between the state nodes and attribute configuration information on each directed edge, and dynamic data modeling is performed so that the constructed state knowledge directed graph can feedback the real change process in the application scenario, thereby improving the interpretability of the data enhancement process; obtain initial data, input the initial data into the state knowledge directed graph, so that the initial data switches between multiple state nodes based on the directed edges, and feedbacks the real change process of the state of the target device; based on each directed edge and its corresponding attribute configuration information, obtain the target data corresponding to the initial data when switching to each state node; when the data volume of the target data meets the preset termination condition, integrate all the target data to obtain the target enhancement data, and the obtained target enhancement data fits the data generated by the real scene, which can improve the accuracy of the additional sample data obtained by data enhancement.
[0016] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 is an optional flow chart of the data enhancement method provided in the embodiment of the present application;
[0018] Figure 2 yes Figure 1 An optional flowchart of step 102;
[0019] Figure 3 is an example diagram of a construction state knowledge directed graph provided in an embodiment of the present application;
[0020] Figure 4 This is another optional flow chart of the data enhancement method provided in the embodiment of the present application;
[0021] Figure 5 yes Figure 1 An optional flowchart of step 103 in FIG.
[0022] Figure 6 is an example diagram of state switching provided by an embodiment of the present application;
[0023] Figure 7 yes Figure 1An optional flowchart of step 105;
[0024] Figure 8 This is another optional flow chart of the data enhancement method provided in the embodiment of the present application;
[0025] Fig. 9 is a schematic diagram of a faulty node provided in an embodiment of the present application;
[0026] Fig.10 This is another optional flow chart of the data enhancement method provided in the embodiment of the present application;
[0027] Fig.11 A schematic diagram of a data enhancement device provided in one embodiment of the present application;
[0028] Fig.12 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0029] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0030] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.
[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0032] In the field of machine learning and data mining, sufficient training data is often required to ensure that the model can learn effective patterns and have good generalization capabilities. However, in actual scenarios, the scarcity or imbalance of real data is a common problem. Data augmentation is a technology for solving data imbalance in machine learning and deep learning. By performing various transformations and operations on existing data sets, new samples that appear different but actually maintain the characteristics of the original data are generated to expand the size and diversity of the data set. The purpose of data augmentation is to reduce the risk of model overfitting and improve the model's ability to generalize to new data without adding additional real data.
[0033] Data enhancement technology is widely used in scenarios where data such as images, text, and speech are unbalanced or insufficient. Taking sensor data as an example, the following are some common and cutting-edge data enhancement solutions: Time series resampling: New time series samples can be generated from existing data by changing the sampling frequency or using time window sliding technology. This includes downsampling to reduce data points, upsampling to increase data density, or smoothing or increasing the fineness of data through interpolation methods. Time offset and misalignment: Shifting time series data on the time axis to simulate data changes at different time points helps the model learn time-related dynamic features. Noise injection: Adding random noise to the original data to simulate natural variations or environmental interference in sensor readings can improve the model's robustness to noise. Signal amplification and compression: Adjusting the amplitude of the signal to simulate signal inputs of different intensities helps the model learn signal features of different scales. Periodic changes: For sensor data with periodic characteristics, data diversity can be increased through phase shifting, periodic repetition, or periodic synthesis. Data synthesis: Use generative models (such as GANs, VAEs) to generate new samples that conform to the distribution of the original data, especially for data types that are difficult to obtain. Feature combination: Combine the different attributes of sensors or the data of multiple sensors to create new feature combinations through feature engineering to enhance the model's understanding of complex situations. Environmental condition simulation: Use mathematical or physical models to simulate data changes under different environmental conditions, such as the impact of temperature and humidity changes on sensor readings. Data standardization and normalization: Although it is not a direct data enhancement, by adjusting the scale and distribution of the data, model training can be made more stable and efficient. Filtering and denoising: Use digital signal processing techniques, such as low-pass filtering and high-pass filtering, to remove or modify specific frequency band components in the data, which can be used as a data preprocessing step or enhancement method to help the model focus on key signals.
[0034] It is understandable that although basic data augmentation methods, such as translation and flipping, can achieve data augmentation, the data obtained is not close to the data generated in real scenarios; advanced data augmentation methods, such as using generative adversarial networks (GANs), variational autoencoders (VAEs) or other generative models for data synthesis, can generate additional samples that are closer to real data than basic data augmentation methods, but they also have the following disadvantages: (1) A complex generative model needs to be trained first. This generative model itself requires a large amount of data to learn the data distribution before it can be used to generate new and realistic data samples; (2) If the generative model is not well established, the generation process is unreasonable or over-generated, it may cause the model to overfit these synthetic samples, reducing the model's generalization ability on real data; (3) The data synthesis process may introduce noise or distort the feature distribution of the original data, especially in high-dimensional feature space, which may mislead the model to learn the wrong pattern. On datasets with fewer features, simple resampling or synthesis techniques may cause excessive information density or information loss; (4) High computational cost: especially for deep learning-based synthesis techniques such as GANs; (5) Synthetic samples lack the diversity and complexity of real data, as well as the randomness and uncertainty in natural data, especially for datasets with nonlinear or complex structures; (6) Reduced interpretability: After data augmentation, the interpretability of the model decision process may be reduced, especially when complex synthesis techniques are used, which weakens the intuitive interpretability of the original data, which is not conducive to the transparency and trustworthiness of the model.
[0035] In related technologies, data augmentation is usually achieved through basic transformations or generative models. However, the data generated by simple basic transformations lacks higher-level features, the training process of the generative model is complicated and the application process is prone to overfitting, which reduces the accuracy of additional sample data obtained by data augmentation.
[0036] Based on this, the embodiments of the present application provide a data enhancement method, device, electronic device and storage medium, which can improve the accuracy of additional sample data obtained by data enhancement.
[0037] The data enhancement method, device, electronic device and storage medium provided in the embodiments of the present application are specifically described by the following embodiments. First, the data enhancement method in the embodiments of the present application is described. The data enhancement method in the embodiments of the present application can be described by the following embodiments.
[0038] Figure 1 is an optional flow chart of the data enhancement method provided in the embodiment of the present application. Figure 1 The method in the embodiment may include but is not limited to steps 101 to 105. Figure 1The order of step 101 to step 105 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.
[0039] Step 101: Obtain historical operation data of the target device.
[0040] The following is a detailed description of step 101.
[0041] The historical operation data includes the operation records of the target device in the past period of time and the expert experience data and industry rules of the target device in various application scenarios.
[0042] Exemplarily, the target device refers to the device or system that needs data enhancement, which can be any type of mechanical equipment, electronic equipment, industrial automation system, medical equipment, etc. These devices will generate a large amount of data during operation, including but not limited to sensor readings, performance indicators, operation records, etc. The operation data of the target device in normal operation and various fault states are collected, which may include but not limited to sensor readings, operation logs, performance indicators, etc.
[0043] It is understandable that historical operation data refers to the operation records and related empirical rules of the target equipment in the past period of time, which may include sensor data, such as physical quantities such as temperature, pressure, speed, vibration, etc., and may also include expert experience data and related rules of experts in related fields on the target equipment in various application scenarios. The collected data is cleaned and preprocessed to ensure the quality of the data. The preprocessing steps may include removing outliers, filling missing values, data standardization or normalization, etc. The cleaned and preprocessed data is stored to obtain historical operation data, which provides necessary information and data support for the next step of constructing the state knowledge directed graph.
[0044] Step 102: Construct a state knowledge directed graph based on historical operation data.
[0045] The state knowledge directed graph includes multiple state nodes, directed edges between the state nodes, and attribute configuration information on each directed edge.
[0046] The following is a detailed description of step 102.
[0047] It can be understood that the state node is based on historical operation data to identify and define the characteristics of the target device in different operating states, and each characteristic corresponds to a state node. For example, the various states of the target device in the actual operation cycle can be used as state nodes. The conversion relationship between the target device in different states is analyzed, and the directed edges between the state nodes are determined based on industry rules and historical data. Attribute configuration information is configured for each directed edge, such as the probability and conditions of state transition. These attribute configuration information reflects the actual possibility of state transition.
[0048] Exemplarily, the state knowledge directed graph integrates state nodes, directed edges and their attribute configuration information to form a complete knowledge graph that can reflect the state changes of the target device, which can more accurately simulate the state changes of the target device in the real operating environment, and the generated data has higher authenticity and generalization ability. At the same time, this method can avoid the overfitting problem that may be caused by data enhancement in the existing technology, and improve the interpretability and reliability of the model.
[0049] See also Figure 2 In some embodiments, constructing a state knowledge directed graph based on historical operation data may include the following steps 201 to 203.
[0050] Step 201: Determine the independent state of the target device in the operation cycle according to the historical operation data.
[0051] Step 202: Establish multiple state nodes based on the operation cycle and independent states.
[0052] Each state node represents an independent state in the operation cycle.
[0053] Step 203: Determine the node attributes under the state node according to the historical operation data.
[0054] The node attributes include the node name, basic attributes and data transformation method of the state node.
[0055] Steps 201 to 203 are described in detail below.
[0056] It can be understood that based on historical operation data, the independent state of the target device in the operation cycle is determined by statistical analysis methods, and based on the determined independent state, a state node is established. Each state node represents an independent state of the device in the operation cycle. Node properties are configured for each state node, including node name, basic properties (such as state duration, state occurrence frequency, etc.) and data transformation methods (such as scaling, translation, rotation, etc.).
[0057] For example, see Figure 3 , Figure 3It is an example diagram of constructing a state knowledge directed graph provided by an embodiment of the present application, showing state node 1, state node 2, state node 3, and state node 4. Different operating states of the target device in the actual operating cycle can be identified from historical operating data by using algorithms such as cluster analysis, as candidates for independent states, and independent states with practical significance are screened out from the candidate states based on expert experience and the physical characteristics of the device operation. For each independent state, a state node is created and a unique identifier is assigned to it. Node attributes are configured for each state node based on expert experience, which can be understood as a class, including but not limited to: Node name: a short name describing the state; basic attributes: such as the duration of the state, frequency of occurrence, correlation with other states, etc.; data transformation method: defines the transformation that the data may undergo in this state, such as linear transformation, nonlinear transformation, etc., that is, a point-like knowledge graph is formed.
[0058] Through the above steps 201 to 203, the independent state of the target device in the operation cycle is determined according to the historical operation data, and multiple state nodes are established based on the operation cycle and the independent state, wherein each state node represents an independent state in the operation cycle, and the node attributes under the state node are determined according to the historical operation data, wherein the node attributes include the node name, basic attributes and data transformation method of the state node, and the node attributes of the state node are derived from expert experience, so that the enhanced data is closer to the real scene, can accurately simulate the behavior of the target device in different states, and improve the interpretability of the data.
[0059] See also Figure 4 In some embodiments, after determining the node attributes under the state node according to the historical operation data, the following steps 301 to 304 may also be included.
[0060] Step 301: Based on historical operation data, determine the transition relationship between state nodes and the transition direction of the transition relationship.
[0061] Step 302: According to the transition relationship between the state nodes and the transition direction of the transition relationship, a directed edge between the state nodes is established.
[0062] Among them, the direction of the directed edge represents the transition direction.
[0063] Step 303: Determine the transition probability and transition condition between the state nodes and directed edges according to the historical operation data.
[0064] Step 304: Generate attribute configuration information on each directed edge according to the transition probability and the transition condition.
[0065] Steps 301 to 304 are described in detail below.
[0066] It is understandable that the industry rules and expert experience data accumulated by the target device in the application scenario are collected and sorted. These data can come from the operation manual, maintenance records, failure case analysis, etc. of the target device. By analyzing the experience rules, the transition relationship between different states of the target device can be identified, which can include the transition in normal operation, the transition when a failure occurs, etc. The direction of each transition relationship is determined, that is, the transition direction from one state node to another state node.
[0067] Exemplarily, for each determined transition relationship, a directed edge is created to connect the corresponding state nodes, ensuring that the direction of the directed edge is consistent with the transition direction determined based on the historical operation data, and establishing directed edges between state nodes in the state knowledge directed graph to ensure that the structure of the state knowledge directed graph clearly reflects the transition logic between different states of the target device. Figure 3 , state node 1 has a certain probability of transitioning to state node 2 and state node 3, state node 2 has a certain probability of transitioning to state node 1, state node 3 has a certain probability of transitioning to state node 4, and state node 4 has a certain probability of transitioning to state node 1.
[0068] It can be understood that based on the statistical analysis of historical data or the evaluation of experts, the state transition probabilities between nodes in different states are learned from the historical operation data, and the transition probability and transition conditions of each directed edge are defined. The transition conditions may include specific environmental parameters, equipment performance indicators reaching specific thresholds, etc., see Figure 3 , associate the transition probability and transition condition with the corresponding directed edges, provide quantitative and qualitative descriptions for state transitions, and generate attribute configuration information on each directed edge.
[0069] It is understandable that attribute configuration information is created for each directed edge, including but not limited to transition probability, conditions to be met, possible triggered events, etc. Utilizing the attribute configuration information of directed edges, the semantic content of the state knowledge directed graph can be further enriched, so that it can more accurately simulate the dynamic changes of the target device state. The rationality and accuracy of the attribute configuration information can also be verified to ensure that they can truly reflect the behavior of the target device in actual application scenarios.
[0070] Through the above steps 301 to 304, a knowledge state directed graph model that reflects the actual change process of the target device state can be constructed. The model is not only highly interpretable, but also can be dynamically adjusted and optimized according to the historical operation data in the actual application scenario to generate samples that are more in line with the real scenario, thereby improving the accuracy of data enhancement and the generalization ability of the model, and reducing the dependence on a large amount of training data.
[0071] Step 103: Acquire initial data, and input the initial data into the state knowledge directed graph, so that the initial data performs state switching between multiple state nodes based on directed edges.
[0072] The following is a detailed description of step 103.
[0073] Exemplarily, the initial data refers to limited real-world data, which can be real-time data collection of the target device at a specific time point, or representative data selected from historical operation data. The initial data can be preprocessed as necessary to ensure the quality and consistency of the data, and at the same time verify the integrity and representativeness of the initial data to ensure that it can reflect the characteristics of the target device in a specific state. Input the initial data into the state knowledge directed graph, input the preprocessed initial data into the state knowledge directed graph as the starting point of the data enhancement process, and perform state switching based on directed edges.
[0074] See also Figure 5 In some embodiments, obtaining initial data and inputting the initial data into a state knowledge directed graph so that the initial data switches states between multiple state nodes based on directed edges may include the following steps 401 to 402.
[0075] Step 401: determine an initial state node in the state nodes, obtain initial data, and input the initial data into the state knowledge directed graph so that the initial data performs state switching based on the initial state node.
[0076] Step 402: Determine the next state node to which the initial state node switches state according to the directed edge and attribute configuration information of the initial state node, so that the initial data switches from the initial state node to the next state node.
[0077] Steps 401 to 402 are described in detail below.
[0078] It can be understood that, based on the characteristics of the initial data, a suitable state node is selected as the initial state node. This state node can be the node closest to the target device state represented by the initial data. The directed edges defined in the state knowledge directed graph and the attribute configuration information of the directed edges (such as transition probability, transition conditions, etc.) are used to determine the next state node of the initial data between the state nodes, that is, the switching path of the initial data.
[0079] Exemplarily, according to the direction and attribute configuration information of the directed edge, the state switching of the initial data between different state nodes is simulated. For each directed edge, according to its attribute configuration information (transition probability, conditions, etc.), the probability of the initial data switching to the next state node is calculated. If the switching condition is met, the initial data is transformed to the new state node as indicated by the edge, and corresponding data adjustments are made according to the attributes of the new state node. The above process is repeated until a predetermined number of state switches are completed or the desired data enhancement effect is achieved.
[0080] Through the above steps 401 to 402, the initial state node is determined in the state node, the initial data is obtained, and the initial data is input into the state knowledge directed graph so that the initial data performs state switching based on the initial state node. According to the directed edge and attribute configuration information of the initial state node, the next state node for the initial state node to perform state switching is determined. The initial data performs state switching in the state knowledge directed graph, which effectively simulates the behavior of the target device in different states and generates data samples that are closer to the actual application scenario.
[0081] Step 104: Based on each directed edge and its corresponding attribute configuration information, the target data corresponding to the initial data when it is switched to each state node is obtained.
[0082] Step 104 is described in detail below.
[0083] For example, see Figure 6 , Figure 6 This is an example diagram of state switching provided in an embodiment of the present application. Figure 6 The state knowledge directed graph in includes state node 1, state node 2, state node 3, and state node 4. The initial data is shown in the figure. State node 1 is selected as the initial state node. Figure 6 The solid arrows in the middle are directed edges of the state knowledge directed graph, and the dotted arrows are the state switching process of the initial data on the state knowledge directed graph. The initial data is transformed according to the node attributes of state node 1 to obtain target data 1, and then transitions from state node 1 to state node 3 according to the directed edges and attribute configuration information. Target data 1 is transformed according to the node attributes of the current state node 3 to obtain target data 2. The state switching continues according to the flow direction of the state knowledge directed graph, transitioning from state node 3 to state node 4. Target data 2 is transformed according to the node attributes of state node 4 to obtain target data 3, and so on. State switching is performed in sequence.
[0084] Step 105: When the data volume of the target data meets the preset termination condition, all the target data are integrated to obtain the target enhanced data.
[0085] The following is a detailed description of step 105.
[0086] It can be understood that by ensuring the data diversity and coverage of the target enhanced data through the preset termination conditions, while controlling the scale of the data enhancement process and avoiding the generation of too much redundant data, the efficiency of data enhancement and the practicality of the target enhanced data can be improved, while ensuring the controllability of the data enhancement process.
[0087] It can be understood that by obtaining target enhancement data through the state knowledge directed graph, there is no need to train a complex generation model in advance, nor is there a problem of requiring a large amount of training data and poor generation model effect. In the process of obtaining additional target enhancement data, no noise will be introduced or the feature distribution of the original initial data will be distorted. The obtained target enhancement data is highly interpretable.
[0088] See also Figure 7 In some embodiments, when the amount of target data meets a preset termination condition, all target data are integrated to obtain target enhanced data, which may include the following steps 501 to 502.
[0089] Steps 501 to 502 are described in detail below.
[0090] Step 501: When the initial data traverses and switches to all state nodes in the state knowledge directed graph, all target data are integrated to obtain target enhanced data.
[0091] Step 502: Alternatively, a preset target data threshold is obtained, and when the total amount of all target data is greater than or equal to the target data threshold, all target data are integrated to obtain target enhanced data.
[0092] It is understandable that in the state knowledge directed graph, the starting point and end condition for the initial data to start traversing can be set. The end condition can be that the initial data traverses and switches to all state nodes in the state knowledge directed graph to ensure that the initial data can traverse all state nodes, that is, the initial data switches from one state node to another state node according to the directed edges and attribute configuration information of the state knowledge directed graph until all state nodes are traversed. In the process of the initial data traversing each state node, the target data obtained after the state transformation is collected, and further integrated to obtain the target enhanced data.
[0093] Alternatively, in some embodiments, the initial data does not necessarily need to traverse all state nodes, but rather needs to go through as many state nodes as possible so that the amount of new samples obtained by data enhancement can reach a preset quantity requirement. Exemplarily, before the data enhancement process begins, a target data threshold can be preset according to the experimental design or application requirements. It can be the number of data points, data diversity indicators, or other related indicators. During the data enhancement process, the total amount of target data generated is monitored in real time, including the number of data points, the types of state nodes covered, etc. When the total amount of target data monitored reaches or exceeds the preset target data threshold, all generated target data are finally integrated to form target enhanced data. The integration can include operations such as removing duplicate data, balancing data distribution, and normalizing data. The integrated target enhanced data is output as the result of data enhancement for model training or other data analysis.
[0094] Through the above steps 101 to 105, the historical operation data of the target device is obtained; a state knowledge directed graph is constructed according to the historical operation data, and the state knowledge directed graph is a knowledge graph including multiple state nodes, directed edges between state nodes, and attribute configuration information on each directed edge, and dynamic data modeling is performed, so that the constructed state knowledge directed graph can feedback the real change process in the application scenario, thereby improving the interpretability of the data enhancement process; initial data is obtained, and the initial data is input into the state knowledge directed graph, so that the initial data switches between multiple state nodes based on the directed edges, and the real change process of the state of the target device is fed back; based on each directed edge and its corresponding attribute configuration information, the target data corresponding to the initial data when switching to each state node is obtained; when the data volume of the target data meets the preset termination condition, all the target data are integrated to obtain the target enhanced data, and the obtained target enhanced data fits the data generated by the real scenario, and can also take into account the randomness and uncertainty in the real data, thereby improving the generalization ability of the model on the real data, and can improve the accuracy of the additional sample data obtained by data enhancement.
[0095] See also Figure 8 In some embodiments, the data enhancement method may further include the following steps 601 to 602.
[0096] Step 601: Generate a fault node based on historical operation data.
[0097] Among them, the fault node represents the fault and abnormal scenario of the target device.
[0098] Step 602: Add the faulty node to the state knowledge directed graph, generate directed edges between the faulty node and the state node, and update the state knowledge directed graph so that the initial data is switched to the faulty node based on the directed edges to generate target data of negative samples.
[0099] Steps 601 to 602 are described in detail below.
[0100] It can be understood that data related to faults and abnormal scenarios are screened out from historical operating data, which may include but are not limited to fault reports, maintenance logs, abnormal performance indicators, etc., and then the screened data is analyzed to extract key features that can characterize faults and abnormal scenarios. Based on the extracted features, fault nodes are generated, and each fault node represents a specific fault mode or abnormal state. Node attributes are configured for each fault node, which may include fault type, severity, scope of impact, etc.
[0101] For example, see Fig. 9 , Fig. 9 It is a schematic diagram of a fault node provided in an embodiment of the present application, showing state node 1, state node 2, state node 3, state node 4 and directed edges between state nodes. Fault node g1 and fault node g2 are added to the state knowledge directed graph to expand the representation capability of the graph, including normal and abnormal state nodes, and directed edges are created between the fault node and the existing state nodes. These edges represent potential transitions from normal state to fault state, and at the same time, based on historical data and expert knowledge, attributes are defined for the newly created directed edges, including conditions and probabilities of transitions. Using the updated state knowledge directed graph, the process of initial data switching to the fault node through the directed edges is simulated to generate target data of negative samples, and the target data of negative samples is used to simulate the behavior of the device in a faulty or abnormal state.
[0102] Through the above steps 601 to 602, a fault node is generated based on the historical operation data, the fault node is added to the state knowledge directed graph, a negative sample injection strategy for fault and abnormal scenarios is provided, a directed edge between the fault node and the state node is generated, and the state knowledge directed graph is updated so that the initial data is switched to the fault node based on the directed edge to generate target data of the negative sample. The ability to generate negative sample data reflecting the fault and abnormal scenarios of the target device is crucial to improving the robustness and anomaly detection capability of the model. It can play a key role in scenarios where negative samples are missing, such as rocket launch data, and at the same time enhances the expressive power of the state knowledge directed graph so that it can more comprehensively simulate the target device.
[0103] See also Fig.10 In some embodiments, the data enhancement method may further include the following steps 701 to 702.
[0104] Step 701: Obtain feedback information of target enhancement data during model training and application.
[0105] The feedback information includes the prediction accuracy on the target enhanced data during model training.
[0106] Step 702: adjusting the node attributes of the state nodes in the state knowledge directed graph, the transition probability and transition conditions between directed edges, the fault nodes or the preset termination conditions according to the feedback information.
[0107] Steps 701 to 702 are described in detail below.
[0108] It can be understood that feedback information refers to feedback indicators used to evaluate the performance of the model on the target enhanced data, such as prediction accuracy, recall rate, F1 score, etc. During the model training and application process, feedback information on the target enhanced data during the model training and application process is collected, especially the performance on the data set that has been processed by data enhancement. The collected feedback information is analyzed to identify the advantages and disadvantages of the model performance, pay attention to the model's ability to identify faults and abnormal scenarios, and based on the results of data analysis, identify problems that may be caused by the data enhancement process, such as overfitting, underfitting, or a high incidence of specific types of errors. The collected feedback information is recorded and stored to provide a basis for subsequent adjustments.
[0109] Exemplarily, the node attributes of the state nodes are adjusted according to the feedback information. For example, if the data generated by a state node causes the model performance to degrade, the node attributes of the state node, such as the transformation method or feature selection, may need to be adjusted. The transition probabilities and transition conditions between the state nodes are updated according to the feedback information to better reflect the performance of the model in actual applications, and the probabilities can be recalculated or the conditions adjusted based on the new data. If the model performs poorly in identifying faults and abnormal scenarios, it may be necessary to add or adjust fault nodes to more accurately simulate these fault scenarios. According to the model performance and the coverage of the data set, the preset termination conditions, such as the target data threshold or the number of traversals, are evaluated and adjusted. The adjusted state knowledge directed graph is used for a new round of data enhancement and model training to form an iterative optimization process to ensure that the adjusted directed graph can effectively improve the generalization ability and prediction accuracy of the model.
[0110] Through the above steps 701 to 702, feedback information of the target enhancement data during model training and application is obtained. According to the feedback information, the node attributes of the state nodes in the state knowledge directed graph, the transition probability and transition conditions between the directed edges, the fault nodes or the preset termination conditions are adjusted, so that the data enhancement process can be dynamically adjusted and optimized. The state knowledge directed graph is easy to modify and expand. If the enhancement effect is not good, the directed graph can be directly modified or expanded without retraining the generated model. It ensures that the generated target enhancement data can continuously improve the performance of the model, and improves the adaptability of data enhancement and the practicality of the model.
[0111] See also Fig.11The present application also provides a data enhancement device 1100, which can implement the above data enhancement method, including:
[0112] The historical data acquisition module 1101 is used to acquire the historical operation data of the target device;
[0113] A state knowledge directed graph construction module 1102 is used to construct a state knowledge directed graph according to the historical operation data, wherein the state knowledge directed graph includes a plurality of state nodes, directed edges between the state nodes, and attribute configuration information on each directed edge;
[0114] A state switching module 1103 is used to obtain initial data and input the initial data into the state knowledge directed graph, so that the initial data performs state switching between the plurality of state nodes based on the directed edges;
[0115] The target data module 1104 is used to obtain the target data corresponding to the initial data when switching to each state node based on each directed edge and the corresponding attribute configuration information;
[0116] The target enhanced data module 1105 is used to integrate all the target data to obtain target enhanced data when the data volume of the target data meets a preset termination condition.
[0117] The specific implementation of the data enhancement device is basically the same as the specific implementation of the above data enhancement method, which will not be repeated here. On the premise of meeting the requirements of the embodiment of the present application, the data enhancement device can also be provided with other functional modules to implement the data enhancement method in the above embodiment.
[0118] The present application also provides an electronic device, including:
[0119] at least one memory;
[0120] at least one processor;
[0121] at least one program;
[0122] The program is stored in the memory, and the processor executes at least one program to implement the data enhancement method implemented in the present application. The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.
[0123] See also Fig.12 , Fig.12 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:
[0124] The processor 1201 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;
[0125] The memory 1202 may be implemented in the form of a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory). The memory 1202 may store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1202, and the processor 1201 calls and executes the data enhancement method of the embodiments of this application;
[0126] Input / output interface 1203, used to implement information input and output;
[0127] The communication interface 1204 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);
[0128] A bus 1205 that transmits information between various components of the device (e.g., the processor 1201, the memory 1202, the input / output interface 1203, and the communication interface 1204);
[0129] The processor 1201 , the memory 1202 , the input / output interface 1203 and the communication interface 1204 are connected to each other in communication within the device via the bus 1205 .
[0130] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the above-mentioned data enhancement method is implemented.
[0131] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0132] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0133] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.
[0134] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0135] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.
[0136] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0137] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c. A), can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0138] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0139] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0140] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0141] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store programs.
[0142] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by those skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.
Claims
1. A data enhancement method, characterized in that: The method comprises: Acquire historical operation data of the target device, wherein the historical operation data includes operation records of the target device in the past period of time and expert experience data and industry rules of the target device in various application scenarios; Constructing a state knowledge directed graph according to the historical operation data, wherein the state knowledge directed graph includes a plurality of state nodes, directed edges between the state nodes, and attribute configuration information on each directed edge; Acquire initial data, and input the initial data into the state knowledge directed graph, so that the initial data performs state switching between the plurality of state nodes based on the directed edges; Based on each of the directed edges and the corresponding attribute configuration information, obtaining target data corresponding to the initial data when switching to each of the state nodes; When the data volume of the target data meets a preset termination condition, all the target data are integrated to obtain target enhanced data.
2. The data enhancement method according to claim 1, characterized in that: The step of constructing a state knowledge directed graph according to the historical operation data includes: Determining, based on the historical operation data, an independent state of the target device in an operation cycle; Based on the operation cycle and the independent state, establish a plurality of state nodes, wherein each of the state nodes represents one of the independent states in the operation cycle; The node attributes under the state node are determined according to the historical operation data, wherein the node attributes include the node name, basic attributes and data transformation mode of the state node.
3. The data enhancement method according to claim 2, characterized in that: After determining the node attributes under the state node according to the historical operation data, the method further includes: Based on the historical operation data, determining the transition relationship between the state nodes and the transition direction of the transition relationship; According to the transition relationship between the state nodes and the transition direction of the transition relationship, establishing the directed edge between the state nodes, wherein the direction of the directed edge represents the transition direction; Determine the transition probability and transition condition between the state node and the directed edge according to the historical operation data; The attribute configuration information on each of the directed edges is generated according to the transition probability and the transition condition.
4. The data enhancement method according to claim 1, characterized in that: The acquiring of initial data, and inputting the initial data into the state knowledge directed graph, so that the initial data performs state switching between the plurality of state nodes based on the directed edges, comprises: Determine an initial state node in the state nodes, obtain initial data, and input the initial data into the state knowledge directed graph so that the initial data performs state switching based on the initial state node; According to the directed edge of the initial state node and the attribute configuration information, a next state node to which the initial state node switches state is determined, so that the initial data switches from the initial state node to the next state node.
5. The data enhancement method according to claim 4, characterized in that: When the amount of the target data meets a preset termination condition, all the target data are integrated to obtain target enhanced data, including: When the initial data traverses and switches to all state nodes in the state knowledge directed graph, all the target data are integrated to obtain target enhanced data; or, A preset target data threshold is obtained, and when the total amount of all the target data is greater than or equal to the target data threshold, all the target data are integrated to obtain target enhanced data.
6. The data enhancement method according to claim 1, characterized in that: The method further comprises: Generate a fault node based on the historical operation data, wherein the fault node represents the fault and abnormal scenario of the target device; The fault node is added to the state knowledge directed graph, a directed edge is generated between the fault node and the state node, and the state knowledge directed graph is updated so that the initial data is switched to the fault node based on the directed edge to generate target data of the negative sample.
7. The data enhancement method according to claim 6, characterized in that: The method further comprises: Acquire feedback information of the target enhancement data during model training and application, wherein the feedback information includes a prediction accuracy rate on the target enhancement data during model training; The node attributes of the state nodes in the state knowledge directed graph, the transition probabilities and transition conditions between the directed edges, the fault nodes or the preset termination conditions are adjusted according to the feedback information.
8. A data enhancement device, characterized in that: include: A historical data acquisition module is used to acquire historical operation data of the target device; A state knowledge directed graph construction module, used to construct a state knowledge directed graph according to the historical operation data, wherein the state knowledge directed graph includes a plurality of state nodes, directed edges between the state nodes, and attribute configuration information on each directed edge; A state switching module, used for acquiring initial data, and inputting the initial data into the state knowledge directed graph, so that the initial data performs state switching between the plurality of state nodes based on the directed edges; A target data module, used for obtaining target data corresponding to the initial data when switching to each state node based on each directed edge and the corresponding attribute configuration information; The target enhancement data module is used to integrate all the target data to obtain target enhancement data when the data volume of the target data meets a preset termination condition.
9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the data enhancement method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data enhancement method according to any one of claims 1 to 7 is implemented.
